Managing Your Content Pipeline

The content and workflow pipeline is something that many people do not give a lot of thought to. Chances are, you build a comfortable rhythm for getting a piece of content into your game, and stick with that as long as you can. I believe pipeline is the most important part of game development. A good pipeline allows for more iteration on content in less time. More iteration means more opportunity to really nail the look of your game, and to increase runtime performance through changes to assets. Better pipeline also makes it much easier to train someone on creating content for your game. If you have a friend who offers to give you a hand for the weekend, you will get much more content from them with a good pipeline. Bad pipeline can also hurt morale without you noticing it. If there is a complicated step, a step with poor feedback, or a step it is easy to make a mistake on, you will have very frustrated content creators. Happy content creators make better games.

If you’re using Unity, chances are most of your pipeline is already much better than in older engines. The simple pipeline is one of my favorite things about Unity. I really like the idea of getting content out of content creation tools as quickly as possible, and having the creators do fine tuning and other settings within the Unity editor. However, there is always opportunity to find ways to improve a pipeline.

I recommend doing a pipeline audit of your game. Write down each content type, and document the process of creating a new piece of that content from scratch, as well as the process of iterating on existing content. If you can, try and include average times for each, and as you step through this process, note any friction points, where users can easily make mistakes, as well as points that take longer than they should. Also document your build process, the steps and how long it takes to get a build of your game onto a device to test things.

I will write up an example of a pipeline audit for creating a new texture for a game.

Step 1: Create texture in image editing tool. 1 hour.

Step 2: Save texture in two places: as a PSD in a working art folder, and as a PNG in the game folder. 1 minute. This is the first friction point in the pipeline: accidently just saving one and not both, you can accidently end up with out of sync art. Export settings when saving to PNG also need to be set, and this is a point that it can be easy to set the wrong settings for the texture type.

Step 3: In Unity, find the texture and apply the import settings. 5 minutes. This is the second point of friction: it can be easy to forget to do this step, and it’s easy to apply the wrong settings.

Step 4: Test texture on an asset in-game on-device. This can be the final asset the texture will apply to, or a generic test material if someone else will be implementing this texture in-game. 20 minutes. Making a build to run on device can take a while. Many content creators treat this as an optional step. I prefer everyone to check all their work on device, it saves people time debugging things later if there is an issue.

Step 5: Commit asset and working art to source control. 10 minutes. This is a problem area for new content, especially if the content creators like to commit many assets at once to source control, maybe the texture is one of many textures associated with a new character, as well as some model files and other assets. It can be very easy to forget individual files within this, especially both the working asset and the in-game game asset.

Complete: Asset is ready to be used in-game.

Lets look at how this process can be improved. Step 2 has quite a few potential problems. Ideally your content creators are working in an image editing tool that can be extended with code to automate things. Photoshop has support for this, and this is a good place for someone who has skill writing Photoshop plug-ins to build a script that will save to the PDF and the PNG at the same time, as well as automatically set export settings for the PNG. This removes user error on the PNG being saved to the wrong location in the build, as well as locks the PDF and PNG file in sync. Having the art in two files like this is bad for pipeline, but in this case is necessary. PDF files store a lot of extra information very useful for creating content, including layers. Unfortunately PDF files are not a format that much tools outside of Photoshop can actually work with, so the export to a format Unity supports, such as PNG, is necessary. If you have a lot of engineering time available, and consider this a priority issue, you can write an asset importer script for PDF files in Unity, and automate a process here. This might even be as simple as writing a file monitoring script that runs in Unity any time any PDF file is changed, which then calls Photoshop in batch mode to export that PDF file to a PNG at that moment.

Step 3 you can automate within Unity through a script that inherits from AssetPostprocessor: and interacts with AssetImporters, such as the TextureImporter I would recommend building some rules to where files are saved in a project, and you can use the folder path of an asset as a hint as to what import settings to apply to it. As an example, if a parent folder of the asset is labeled “UI”, you can apply texture settings that are specific to UI textures using these. I would recommend leaving a way for users to override these settings, perhaps by marking the asset with a label, sometimes there are cases where the settings for an asset needs to be tweaked manually. If you build a solution like this for step 3, then you can remove step 3 entirely from the pipeline.

The fourth step I have listed most people treat as optional, especially for content like textures. While game development often feels like you are sprinting to completion, taking some time to actually check your work running on device will save so many headaches later. You might discover that the mipmap settings for the texture are causing it to display at a much lower resolution than you intended on device, and fix that before anyone else sees it. If you did not check on device, then it might be a few days before someone does, and people might not even comment on it, the testing and engineering staff who most often run on device might just assume you meant for that texture to display at 32×32 instead of the 512×512 you created it at. The original content creator for an asset is going to be the person best able to know what that asset is supposed to look like within the game. Checking every asset individually might be too time consuming, especially if the content creators do not have full access to devices, so it might be worth maintaining a list of content created since the last check, and every few days load the game up on device to check every asset since the last check.

The fifth step is a huge point of concern, and very difficult to address. A common problem that crops up in games is a content creator has only been checking in the game asset, and not the working asset. At some point access to the working asset gets lost, either someone else needs to work on the piece of content, or mistakes are made editing the working asset and there is no point to roll back to. It is also easy to forget individual assets to check in when doing a larger commit to source control. If a new character model is added, this probably includes source and game art for the model file, source and game art for multiple textures, and associated meta files within Unity that contain the import settings for these assets. The solution that seems to work best here is full source control integration to each point in the pipeline. Setting up the content creation tool, such as Photoshop, to communicate with your source control solution to mark new assets to be added, and automatically check out existing assets that content creators attempt to change, goes a long way towards content creators forgetting to check in files. Validation scripts that run on your source control server can also help catch problems before they are committed. Most modern source control solutions, such as Perforce, allow you to run code to validate check-ins before they occur. One such validation step you might write for your source control server is to reject any asset file check-ins that do not have an associated meta file in the check-in. Another might be to reject any source art or game art check-ins without the other piece of content, forcing them to be checked in in lockstep.

In an ideal world, you have now had all the time you needed to correct and work on your pipeline, let’s look at what the pipeline for getting a new texture into your game is now.

Step 1: Creature texture in image editing tool. 1 hour.

Step 2: Save texture. Automation scripts in the image editing tool ask the user for a path for the final game art, and save a PNG file there, and then automatically generate a path to save the working art PDF file outside of the Unity assets folder. Another set of automation scripts then tell source control to mark these files as new files to be committed, and place them in a changelist together. 1 minute. The friction at this point in the pipeline has been removed, there is no more room for user error. Of course, bugs in the tool can crop up here, so keep a close eye on the process until you’re confident it all works properly.

Step 3: No more step 3. The import settings are automatically figured out by a script that runs in Unity when Unity sees the new texture file. Another set of scripts that fire off when meta files change or are created calls the command line options for source control to add that meta file to source control.

Step 4: Content creator does a quick test in Unity editor to look for any major errors, and then adds the texture to an asset list to verify on device at the end of the week when they have access to a device. 5 minutes. Not my ideal of checking every asset on device every time, but this compromise will keep the content creators from getting frustrated from waiting in line to use the test machine to make a build.

Step 5: When the user tabs over to their source control client, a changelist is all set up with the new asset’s working art, game art, and meta file. They can type a quick check-in note and commit the new assets. 5 minutes. If the content creator forgot to open Unity, and the meta file is not in the changelist, then when they attempt to commit the change the source control client pops up an error message asking them to add the meta file to the changelist.

This new pipeline is a lot cleaner, and removes most user error from the process. One of the reasons I fell into tools development was I found making changes and updates like this extremely rewarding. Your clients become the people you directly work with, and you can get feedback and work directly with them to improve how much content they can create in a week. This will then have a direct impact on the total content in the game, and you will be able to look at the amount of content in this game versus the previous game you might have developed, and see a huge increase in content with minimal change to staff.

More astute readers probably noticed a problem with all the new technology introduced above: You have just added many steps to the pipeline for adding a new content creator to your project. They now need to install these Photoshop plugins before they can work effectively. If your project has new content creators joining constantly throughout the project, then it is worth building a tool for managing your pipeline plug-ins. This is a lot of engineering work, and only really useful for teams with a lot of engineering time available, and a steady stream of new content creators. To solve this, you can build a project setup tool you have users run before beginning work. This tool can then install or update the plugins needed for all of the tools to create content for your game, and even ensure that the correct version of Maya, Max, Photoshop, and whatever content creation tools are installed, as well as third party libraries such as Python.

A final thought on pipeline today is one direction versus bi-directional pipeline. In yesterday’s post on localization, I covered this briefly. Often times content creators will prefer to make changes at the “game art” level, and not the “source art” level, for localization this would be changing the text database in the database editor, instead of in the spreadsheet file and doing an export. Bi-directional pipelines like this can save content creators time, they can iterate on the game art level rapidly, and then apply those changes to the source art when they are ready to commit to source control. I do not like a pipeline like this because it adds a huge amount of engineering work to support changes going from game art back to source art. It also adds complication to the process where you might have to resolve changes made to the source and game art at the same time. I would much rather spend my time addressing a separate issue: what is wrong with the pipeline of working in the source art that content creators would rather work in the game art directly? Improving the one directional pipeline might give quicker, better results. This is obviously a hot topic in pipeline discussion, so I would suggest discussing it thoroughly with your team before making a commitment either way, someone might have a great reason to want to make certain changes to the game content, and then rolling those up to the source art.

A large part of improving the performance, and quality of your game is going to involve content iteration. Keeping a tight leash on your pipeline will give you much more high quality content for less time invested by the content creators. This is especially great for your customers, while it’s hard to market “Our game is more fun because we could make changes easily to our project”, the quality of your output should show this instead, and the players will just have a better time playing your game, and will hopefully come back for the sequel.


Joseph Stankowicz is a software engineer who has worked in the video games industry for over eight years. The last two years have had a heavy focus on Unity development, where he helped ship over eleven titles to iOS and Android platforms. He also is really excited about 3D printing, and keeps his Solidoodle 3 printing out stuff as often as possible. You can view his LinkedIn profile here

Tagged with:
Posted in Unity3D, Unity3D Performance
2 comments on “Managing Your Content Pipeline
  1. J.C. Connors says:

    This is awesome, Joe. Now talk about how this changes when your Unity game needs to do dynamic updates from a server somewhere.

  2. That’s a great idea. Dynamic content and fallback systems are the sort of thing that can really reduce the overall load times of games. That will be the topic of my next Unity post.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: