The curious case of environment variables
BAML is a fundamentally different way to interact with LLMs through a new lightweight programming language which offers structured outputs from LLMs. But if you're already familiar with BAML, you might have noticed that environment variables were a bit tricky to work with. Here's an example of how it was a hassle in the past:
from baml_client import b
from dotenv import load_dotenv
# This will work since load_dotenv is imported after baml_client
load_dotenv()from dotenv import load_dotenv
from baml_client import b
# This will not work as expected since baml_client is imported AFTER load_dotenv
load_dotenv()Even if you were careful about the import order, simple things like changing an environment variable right before a Baml function call would not work as expected.
import os
os.environ["OPENAI_API_KEY"] = "old-key"
from baml_client import b
b.ExtractResume(
resume_url="https://www.linkedin.com/in/ba11b0y/"
)
os.environ["OPENAI_API_KEY"] = "new-key"
# This will still use the old key!
b.ExtractResume(
resume_url="https://www.linkedin.com/in/ba11b0y/"
)
You see what's happening here?
Some background
BAML uses Rust internally, both to spawn a runtime and to generate client libraries in different languages.
Each programming language has its own runtime and its own way of reading environment variables.
For example, in Python, you can read environment variables using the os module.
import os
os.environ["OPENAI_API_KEY"] = "key-123"While, in Typescript, you can read environment variables using the process.env object.
process.env.OPENAI_API_KEY = "key-123"But these changes pertain to the language runtime and not the BAML runtime.
So why not use Rust's std::env::var() to read them into the BAML runtime?
One might think that we could easily fix the above problem by making sure that the BAML runtime always reads the environment variables using Rust's own std::env::var() and then
use them while calling BAML functions. But there's a catch here!
BAML's runtime is initialized during the initialization of the language runtime and that means BAML runtime would not be aware of the environment variables that you set in your code since those are modified in your language's runtime. This is a big problem because you could change environment variables anytime in your code, and the client code generated by BAML would not be aware of it.
Jump to the fix already!
Well, the fix turned out to be a quite a bit of refactoring to lazily inject environment variables into the BAML runtime right before every function call. This involved a lot of changes to the BAML runtime and the client libraries generated by BAML, so we can read in environment variables from your respective language's runtime into the BAML runtime. Now that means no matter the order of imports of your environment variables library, or setting environment variables in your code, the BAML runtime will always use the latest environment variables.
from baml_client import b
from dotenv import load_dotenv
load_dotenv()
b.ExtractResume(
resume_url="https://www.linkedin.com/in/ba11b0y/"
)
os.environ["OPENAI_API_KEY"] = "new-key"
# This will use the new key!
b.ExtractResume(
resume_url="https://www.linkedin.com/in/ba11b0y/"
)As an enhancement, we've also added support for passing in environment variables per baml client using baml_options.
from baml_client import b
env = {"BAML_LOG": "DEBUG", "OPENAI_API_KEY": "key-123"}
# Create client with environment variables
my_client = b.with_options(env=env)
# Uses the environment variables
result = my_client.ExtractResume("...")That's it for this post! Now you know environment variables are not as simple as they seem :)
If you're a curious mind, here's some further reading and if this still doesn't satiate your curiosity, you can always reach out to us on Discord or Twitter.