Linux Environment Management

UPDATE: make sure to check out the sequel to this post for the best solution I have found

Linux Environment Considerations

The Linux Environments that ASIC and SoC (chip) design teams use are often messy and confusing. When team members work on multiple ASIC projects that each require different sets of tools the problem is even worse. When engineers spend time fighting the environment that slows down the development of our chips little by little each day. This doesn't need to be the case. This post explains:

• What a Linux Environment is
  
• Why it's important, especially for ASIC projects
  
• Techniques to configure and manage the environment
  

What is an Environment?

In Linux, every process runs with a set of environment variables available to it. This set of environment variables is often referred to simply as, the environment. Here are some examples of how programs use the environment:

• The command-line shell uses the PATH environment variable to find the programs you ask it to run
  
• Programs use LD_LIBRARY_PATH to find compiled libraries that they rely on
  
• ASIC design and verification tools use the LM_LICENSE_FILE environment variable to determine how to contact their required license servers.
  

For most Linux users the environment isn't much of a concern. When they log in it gets configured by shell initialization files for the common programs and libraries that they use and they are good to go. We ASIC engineers are much more demanding of our environment. We generally use a wide array of software tools that are not included in our Linux distribution. We also keep multiple versions of each of those tools installed so we can try new versions out and revert back to using old versions when needed. Our environment needs to be configured for each of these tools and reconfigured when we want to switch which version of the tool we are using. Making matters worse most of these ASIC tools require more than just PATH and LM_LICENSE_FILE environment variables, they have a wide assortment of other variables they expect to be set in your environment for proper operation.

Managing The Environment

There are several ways to manage your Linux shell environment. Let's take a look at them.

Default Shell Initialization Files

This was mentioned in the introduction. With this technique you simply put all configuration in the shell initialization files (~/.profile, ~/.bashrc, ~/.cshrc, etc.).

Benefits:

• This is the standard way of managing your environment in linux
  
• Simple, easy to understand for everyone
  
• You can use standard shell commands to inspect your environment: env, echo $VARIABLE_NAME, etc.
  

Downsides:

• Only supports one version of any given tool. To use a different version of a tool you have to edit your shell initialization files, then start a new shell for them to take affect.
  
• Everyone has their own initialization files, which makes it hard to ensure everyone is using the same environment
  

Explicit Environment Files

Another approach is to have explicit environment configuration files. When you log in or start a new terminal session a minimal environment will be configured by the normal shell initialization files, and then you issue a command to configure that shell instance with the desired environment. The environment configuration files can be simple shell-syntax files (and the command as simple as . environment-init or source environment-init). Alternatively, there is an open source tool named Environment Modules that teams often use for this.

Benefits:

• Environment configuration files can be centralized so there is one file that everyone uses
  
• You can maintain multiple configuration files as needed: one per tool version, one per project, one per engineering role, etc.
  
• Project leads and/or tool administrators can easily create and maintain the configuration files so individual engineers don't have to
  
• You can use standard shell commands to inspect your environment: env, echo $VARIABLE_NAME, etc.
  

Downsides:

• Once an environment configuration is loaded it's difficult to unload (you need to start a fresh shell to be sure)¹
  
• If you use Environment Modules for this, environment configuration files have to be written in Tcl²
  

Per-command Environment Files

Another approach is to prefix every command that needs a special environment with a command that spawns a subshell, sets up the necessary environment, and then runs the intended command. It looks sort of like this:

envA simulation-command

This way your interactive shell is never poluted with project- or tool-specific settings (just the subshell is) and you can easily switch to a different environment on a per-command basis:

envA simulation-command
cd <another-project-area>
envB synthesis-command

Benefits:

• All the same benefits of Explicit Environment Files mentioned above
  
• Easy to use different environment configurations, even on a per-command basis (you don't have to start a fresh shell for each new environment)
  

Downsides:

• Sometimes hard to remember to prefix every command
  
• If you don't often use different environments the prefix feels like unnecessary awkwardness
  
• Inspecting the per-command shell environment is not as simple as typing echo $VARIABLE_NAME or env, you have to do something like envA sh -c 'echo $PATH' or envA env
  

Smart Environment Manager Tool

This is basically the same as using Explicit Environment Files above, but instead of a simple source command or Environment Modules, you can use a tool that has the ability to load an environment and to safely and completely undo (unload) an environment configuration when you want to switch from one environment to another. An open source tool that does this is named albion (full disclosure: I wrote albion). Using it looks like this:

albion env projectA
simulation-command
albion env projectB
synthesis-command

Benefits:

• All the same benefits of Explicit Environment Files mentioned above
  
• Easy to use different environment configurations and switch between them
  
• Environment configuration files use sh syntax, not Tcl
  

Downsides:

• You can't switch environments in a single command like Per-command Environment Files allows you to, but a future version of albion could support this
  
• albion is still somewhat new and might need a little work or customization to fit your specific needs
  

Conclusion

A messy Linux environment can be confusing to engineers and slow down a project. With some thought and use of a good tool the Linux environment can be tamed. A tame environment will make your engineers happier and your project will go smoothly and more quickly.

Footnotes:

1. Environment Modules claims it can cleanly undo (unload, in their terminology) an environment by simply inverting every command in the modulefile (e.g., setting a variable becomes unsetting the variable). If someone has removed or changed a command in the modulefile or deleted it altogether in the time after you loaded it this technique obviously does not work.

2. If this seems OK to you, consider that most ASIC tools provide you with an environment configuration file in csh or sh syntax that you will then have to translate into Tcl, for every version of each tool you install.