A Vocabulary for Diagrams

Data and Processes

Data — such as Files, Assets, etc. — are drawn as simple rectangles. Data are located on the diagram based on representation and revision status.

Processes — (including Applications and Plugins) are drawn as slanted rectangles. Processes are located as needed, near the data they interact with. (In other words, locate by usage, not by where the binary file lives.)

Note: on a whiteboard, slanted rectangles are way less sloppy than circles, and after a few false starts, very easy to draw.

Axes

Pipelines do two big things for us in digital production. First, they manage the conversion of data formats, and second, they manage the synchronization of data sets across infrastructure levels. Examples of the latter include Perforce syncing and cloud rendering.

Horizontal Axis — represents data conversions. In whiteboard sessions we find it most natural to draw conversions left to right, just as in the above diagram.

Vertical Axis — represents data synchonization. In whiteboard sessions we find it most natural to draw global datastores and cloud services at the top of the board, and local stuff lower on the board.

Example:

Scopes

Any distinct infrastucture level like Cloud, Server, Asset Server, Perforce Server, Local Workspace, etc. gets drawn in its own horizontal band.

This diagram shows the skeleton of a typical pipeline: store at server level, sync to local, process, sync to cloud, process in cloud, sync to server.

Structure

Often files or processes need grouping.

Applications and Source Files

Instead of inputs on left, outputs on right—with a corresponding change in representation—we draw the application as loading and saving the same file.

Note the representation doesn't change during load and save, hence the vertical relationship. This also mirrors the way caches are handled, below.

I like to draw applications a little bigger. This gives them a visual weight which illustrates where the artists spend their time.

Application Import and Export

Plugins

Applications (and Processes in general) can have Plugins.

Often it's important to represent the contents of a file, i.e. does it contain geometry or animation etc. It's most whiteboard friendly to indicate content with little colored badges, because you can add and remove badges without a lot of erasing.

This Asset has components for Model Structure, Geo, Rig, Shader Binding, Shading, and Texture.

The badges help us stummarize complex structure.

File References

Files often refer to other files, via include, import, load or other equivalent syntax. This is indicated as follows:

Using lines to indicate references generates way too much visual clutter:

• references can come from anywhere — often from far earlier in the pipeline;

• also, as files propagate they often carry the references with them.

Overrides

File references are often resolved via an override mechanism. This is indicated with a diamond, as an homage to the classic flow chart decision symbol.

By convention, the reference in the lowest scope wins. The following diagram shows a reference which resolves to Upstream File 2.

Indirection

Some define indirection to be the same as reference. A more common understanding of the term is reference to reference. That's what I'll use throughout this web/book.

At first glance this is similar to override. But a deeper look shows this to be very different.

• Override — evaluate a list of options, and return the first one which works.

• Indirection — provide a pointer into a list of equivalent options.

The above diagram shows two references: blue and green.

• The green (diamond) reference implements override. The override tries the local version of CoffeeCup, then the server version.

• The blue/green combination reference on the CoffeeCup node implements indirection. If the CoffeeCup override falls through to Server, it is pinned (via indirection) to Revision 3 there.

References and Content

The Content section (above) has a diagram showing an Asset with badges for internal content.

That asset has a lot of internal references which were not originally drawn, but if we did draw them:

i.e. the Model Structure refers to Geo, Rig, and Shader Bindings; and the Shader Bindings refer to a Shader; and the Shader refers to a Texture.

This is a common structure, but this drawing is hard to draw and read. We need a simpler diagram:

This is a very useful diagramming shortcut:

• left-pointing triangles represent internal references within a structure;

• large triangles with stems represent references which are more widely separated.

Both are equivalent. The downstream (referring) file contains a pointer to the upstream file (the file referred to), via include, import, load, or other construct.

Note: the Shader Binding is an Indirection Layer to the Shader. and the Shader is an Indirection Layer to the Texture. (That is certainly correct, i.e. the diagramming rules are consistent.)

Aggregation is where one file is simply a list of references to other files. For example, a Set is a list of Models.

There are a few situations that need special handling.

File Drawn in Multiple Places

Imagine we are precalculating the animation poses for a shot. For an explanation, see the Bake Pattern. Prior to calculation, our shot looks like:

And after caclulation:

We need to draw the Shot structure as both input to and output from a procedure. The solution is a dashed line, no arrowheads:

The following three diagrams are equivalent:

Directionality

Synchronization typically runs both ways, which can lead to lots of arrows.

To be specific, the synchronization/revision data flow is:

First vertical arrow (reading left to right) — Server data is checked out to the local workspace, which changes the local revision level to match the server.

Second vertical arrow — The data is loaded into the application, which syncs the internal state of the app to match the revision level.

Third vertical arrow — After being modified in the application (which bumps the revision level internally), the data is saved, which bumps the local revision level.

Note this new revision level has not been explicitly named or numbered. This will happen on checkin. For now, the source file is newer than what is on the server.

Fourth and last vertical arrow — Finally, the data is checked in or committed, which bumps the revision level on the server.

I prefer to elide some of these arrows:

Note: In each case, the arrows terminate on the Source Control node, rather than the file itself. This represents the idea that check-in/out and sync operations engage with the source control system, not with the internal storage of the file itself.

Synchronization Processes

Typically there are are processes which manage the vertical syncing between scopes, (such as Perforce, Git, FTP, etc.) but I prefer to elide them for readability.

If we include them, the graph can get wide very quickly:

The following pipeline illustrates most of these concepts.

Reading from left to right:

Geometry — The source file is stored at server level, synced down to an artist workspace, edited, and converted via plugin.

Handoff — The converted file is synced up to server so it can be synced down to a different artist, who works on...

UVs and Textures — The raw geo is imported after handoff. In addition, the UV and Texture source files are stored at server level and synced down to this workspace. After editing, the data is exported via plugin to a local asset structure. Once approved there, it is checked in

Asset — is rebuilt/installed at server level, where it is available for referencing via an indirection scheme.

Shot Creation — Again, the master shot manifest is stored at server level, synced down to a local workspace. Asset references are via override, so if the artist has his/her own local version, he/she will pick that up.