Butane (formerly the Fedora CoreOS Config Transpiler, or FCCT) is a command-line utility that is used to generate human readable configuration into Ignition configuration files. Butane is an open source repo maintained by CoreOS, and contributed to and enhanced by many. If you take a look at the Butane repo and look in the Config folder, you can almost see some of the stakeholders. These are configurations which are used to target different platforms (i.e., FCOS, Flatcar Container Linux, Red Hat OpenShift, etc.) Each of these configurations will have a stable and experimental phase that also targets stable and experimental specs of Ignition. I would describe these as levers that a contributor can exercise to impact the functionality they want to expose to the end users of these target platforms.
What do we mean by sugar?
So what is this sugar, and why do we want it? With the primary focus of Butane being the translation of something that is easily understood to something that is more specific and less easily understood, honestly, you could view all of Butane as some form of sugar. I like to think of it as magic dust we use to make the user experience more enjoyable by reducing the amount of effort it takes to do common tasks. Sugar typically impacts all target platforms, and is added to the experimental spec, which gets stabilized eventually.
Example use case
As a user, if I want to create a file which is nested in a few directories (i.e.: /foo/bar/directory/myfile.txt) with a specific permission set, I do not want to have to describe each independent folder the file is nested in (i.e.: /foo, /foo/bar, /foo/bar/directory). Currently in Butane, you would have:
variant: fcos
version: 1.6.0-experimental
storage:
files:
- path: /foo/bar/directory/myfile.txt
user:
name: builder
contents:
inline: my content!
directories:
- path: /foo/bar/directory/
user:
name: builder
group:
name: builder
- path: /foo/bar
user:
name: builder
group:
name: builder
- path: /foo
user:
name: builder
group:
name: builder
As you can see in this example Config, there is already a lot of bloating and potential surface area that could expose the user to user error and large Butane files.
Steps to add sugar
UX changes
The first thing you want to do is decide what type of user experience you want to provide the consumer. So looking at this and the problem provided, we know we want to eliminate the need to add configuration for the individual directories. We want some magical dust to be able to interpret what the user wants. Since the issue stems from the file being in a nested directory, lets look at a file's fields.
Find what part of the translation is too bitter
Looking at the latest spec, we can see that we have already defined a custom translator for file and registered it here. Initially, since we are adding sugar to files, you might think that our work belongs in the custom translator of files. However, since our sugar is going to be expanding directories, we need to place our sugar at a point in translation which the sugar can interact both with files and directories. Since we know we need to interact with both files and directories we can look at our schema, and find the earliest struct which contains and knows about both of them. In this case storage knows about both of them; you can see that here.
Add some sugar!
Info alert: Note
At this point, please know that the following code is condensed and summarized to reduce bloating. I will try and focus on the important bits.
Now that we know our storage struct is too bitter, we can go about adding the sugar to make it oh-so-sweet. To do this, we can create a custom translator similar to the custom file translator found here, but in this case storage will register the custom file translator as well. To do this, understand we are creating a map from Butane's struct to Ignition's. And we now have the chance to make changes to the outputted Ignition config. So our signature will look like this:
func translateStorage(from Storage, options common.TranslateOptions) (to types.Storage, tm translate.TranslationSet, r report.Report)
Now we start defining our translator, we are translating from YAML to JSON.
tr := translate.NewTranslator("yaml", "json", options)
And we have some pre-existing custom translators that we can register like this:
...
tr.AddCustomTranslator(translateDirectory)
...
Now we can start translating from the Butane structs to Ignition. So looking at our storage struct we can see all the fields we need to add translation for. To do this, we start adding field by field, declaring the top most field our prefix:
...
tm, r = translate.Prefixed(tr, "directories", &from.Directories, &to.Directories)
translate.MergeP(tr, tm, &r, "disks", &from.Disks, &to.Disks)
...
}
Awesome! We are now at the point where our translator should handle all the requirements for converting from YAML to JSON between Butane and Ignition. The last part is adding our sugar.
To do this, we can now work against the from object and iterate through our files. While iterating through the files, our sugar will look at the parent to see if action is required.
Once we have decided that a file has directories that need to be expanded, we can then:
// For each directory that needs to be expanded based off parent and file path
// We create the Ignition struct of the directory
renderedDir := types.Directory {
Node: types.Node{
Path: dir,
Group: types.NodeGroup{ID: file.Group.ID, Name: file.Group.Name},
User: types.NodeUser{ID: file.User.ID, Name: file.User.Name},
},
DirectoryEmbedded1: types.DirectoryEmbedded1{
Mode: file.Parent.Mode,
},
}
// Append it to the return object `to`
to.Directories = append(to.Directories, renderedDir)
}
Lastly, we need to do some housekeeping and update our report by associating all these directories to a context path by having a line similar to the following:
tm.AddFromCommonSource(yamlPath, path.New("json", "directories"), to.Directories)
Finally, we have created sugar and a custom translator. Now, let's register it.
In the main translate function, we can register our custom translator like so:
tr.AddCustomTranslator(translateStorage)
We can see this code work by creating some tests similar to the directory tests here.
Updating documentation
Ensuring that the documentation accurately reflects the changes made to Butane's functionality is crucial for maintaining clarity and transparency. You can see the long list of documentation here. This is every platform Config type that Butane supports. They are subsets of functionality but as a result they have a large overlap of documentation. Thankfully we don't need to manually update the docs on each of these targets, there is automation built in!
1. Update the docs related to the struct
Begin by including your new fields within the structured documentation file, accessible here. As these additions are in the file, navigate to this specific section within and add something like the following:
- name: files
children:
- name: contents
children:
- name: source
transforms:
- regex: "Supported schemes are .* haven't been modified."
replacement: 'Only the [`data`](https://tools.ietf.org/html/rfc2397) scheme is supported.'
if:
- variant: openshift
- name: mode
use: mode
- name: parent
after: $
desc: The parent directory for the specified file. By declaring a parent, directories from the parent to the file's target destination are automatically created.
children:
- name: path
desc: The path of the directory within the file's 'path'.
- name: mode
desc: Directory modes are set to 0755 as a default if not specified, and the directory does not exist prior to the specified file.
Once you've updated the YAML file, execute the ./generate command from the root directory of the Butane repository. This action triggers the automatic regeneration of all platform configuration documents, ensuring that they are synchronized with the latest information.
2. Update examples in examples.md
Finish the documentation update process by adding examples to the examples.md file. Here, you can include a a demonstration of the new sugar. Provide clear instructions and examples to assist users in understanding and using the sugar.
Final thoughts
The concept of sugar in Butane is as varied and nuanced as the use cases it addresses. It can be as simple as adding a touch of convenience to a common task, or as complex as restructuring core functionalities. Understanding its potential and implications can empower us to enhance Butane's day-to-day user experience significantly.
