Using Maven to handle Unity assets with Git or SVN

Alright. Let’s talk version control.

If you have ever tried to use Git or SVN with a project that contains large files (models, textures, libraries, etc), you’ll know that Git and SVN choke on them.  They’re made to handle text files of code, and uploading and downloading large binaries just slows the system to a crawl.

In Java, we have the same problem with a ton of jar libraries clogging up the the version control. However, a solution to this exists in the java world called Maven Repository.  A Maven repo is designed to handle binaries, and it’s commonly used with build tools (like Ant, Gradle, or Maven pom) to supply said binaries at run time or on demand.  This allows you to only check in source code to your code repo and let the Maven repo do the heavy lifting of binaries.

Maven is definitely able to handle Unity assets, but it isn’t perfect.

Pros:

  • Doesn’t clog up version control – a simple xml file is used to fetch assets from the maven repo
  • Assets can be version-ed
  • Asset dependency is handled automatically
  • Can easily handle several gigs worth of data
  • Free and Open source!

Cons:

  • It was designed for Java – so it doesn’t integrate into Unity
  • You have to run your own private repo to host Unity assets
    • You have to package and upload assets bundles
  • Custom assets still have to be checked into the code repo

Let’s get started.

Unity:
Go to Edit>Project Settings > Editor Settings and set Version Control mode to “Visible Meta Files” and Asset Serialization Mode to “Force Text.”

In your Assets folder, create a folder called “maven”.  You can call this whatever you want, or move it anywhere (or even exclude it), but you will have to modify your POM file accordingly.

In the root folder of your project, create a text file called “pom.xml” and populate it like so:

 

4.0.0
com.mycompany.app
my-app
1

org.apache.maven.plugins
maven-dependency-plugin
2.8
true
true
Assets/maven

 

 

 

You can change the company name, artifact, and version if you want, but it isn’t necessary.  Maven stores all binaries with a suffix of version and classifier; the predefined pom above assumes you want to strip that info from the asset’s base folder name. This can be toggled via the stripVersion and stripClassifier variables.

Create a second Unity project with the same settings as the first one. I’ll call this project Unity Deployment project from now on. We’ll use this one to create and upload asset packages to the maven repo. In the root folder of the project folder, create an assembly.xml file and populate it like so:
asset

zip

Assets/${asset.name}

*
**/*

 

This is what tells maven which files to grab and where. It’s currently set to grab everything from a folder specified in the main pom under “asset.name”.  Speaking of which, create a pom.xml in the root folder and populate it like so:

 

4.0.0
rootFolderName
UTF-8

packagenamehere
artifactnamehere
versionhere
jar

name of asset here
url to asset store here

 
${asset.name}


org.apache.maven.plugins
maven-assembly-plugin
2.2.2

assembly.xml

make-assembly
package

single

 

 

 

 

 

 

Maven:
Install maven and add it to your system path.

Figure out a hosting solution. You can either run a repository manager like Apache Arichiva or JFrog’s Artifactory which will give you fine control over asset storage and management, use a FTP server such as FileZilla, or just designate a file system location such as a network drive or Dropbox for the repo.

Once you have your hosting solution figured out, you will need to create an xml tag for it :
unityRepo
http://locationToYourRepo

 

Place this in the repository section of both pom.xml files as well as the distributionManagement section in the Unity Deployment project’s pom file for upload. If your repository requires a username and password, go to the maven settings file (~/.m2/settings.xml or C:\Users\username\.m2\settings.xml ) and add a server block in the “servers” section

unityRepo
username
password

Deploying Assets:
Hokay, now that the ground work is laid, it’s time to actually start doing things. Open up your Unity Deployment project and import an asset that you want to Maven-ify. Once it’s done, close Unity and open up the pom.xml file in the project’s root directory., and modify the asset.name, groupId, artifactId, version, name, and url. Here’s an example using Unity-Chan.

 

UnityChan
UTF-8

unity.com.unity-chan
unitychan
1.1
jar

Unity-Chan model
https://www.assetstore.unity3d.com/en/#!/content/18705

You can put anything you want in the groupId and artifactId, but as far as naming conventions go, “groupid” is broken down into “com.company.product”, and “artifactId” is the specific module.  For Unity assets, I like to prefix the groupId with “unity” so as not to get things mixed together with the java binaries in our maven repo. Also, “com” is for commercial and “org” is for open source/non-commercial/etc. If the asset is just a singular piece, then you might just leave the groupId at the company name (like the above unity-chan example).  But I digress, it is your repository, so it is your personal preference. This is just for organization.

Next, open up a command prompt in the project’s root directory and run:

mvn clean package

This will make maven process the pom (and accompanying assembly) files and package them, but not upload them. Maven should have created a “target” folder. Inside there you should see a few miscellaneous folders (for maven’s various tools) along side a jar file and a zip file.  In the above example, they would be called “unitychan.jar” and “unitychan-asset.zip”.  You can safely ignore the jar file. As mentioned earlier, Maven was designed for Java, and as such it tries to package up java code. However, since we’re not using a java project, it just creates a jar file that only contains the pom file.

The zip file is what we’re really here for. Open it up and make sure that it got everything you wanted. When maven grabs the dependencies, it will extract it exactly like that.

If everything looks good, run

mvn deploy

from the command prompt to have maven upload this to your repo.

Retrieving Assets
Time to see if all of our work paid off. Go back to our first Unity project, the one we want to pull the assets into from our repo.  Open up the pom file and add a dependency tag for our asset in the dependencies section. This is and should be a copy past from the pom file we used to upload the asset. The only exception is that we have to add the classifier:asset tag and the type:zip tag. For example:


unity.com.unity-chan
unitychan
1.1
asset
zip

 

Now open up a new command prompt in this project’s root directory and run

mvn clean dependency:unpack-dependencies

This tells Maven to pull down the dependencies listed in the pom file and unpack them.  By default, it puts them in the target folder with the classifier and version appended to the directory. However, we overrode that in the configuration section of the pom file.

Once it’s done. Open up the project with Unity and check to make sure everything imported correctly.

It should be noted, that Maven will not delete anything. If you need to re-download assets for whatever reason, just delete everything inside the maven folder and let it handle it. The reason for this is that Maven likes to use the “target” folder for it’s work area, and it will happily delete and modify anything in there. When you tell it to put things outside of the “target” folder, it plays it safe and won’t delete/overwrite anything.

Version Control
To make things easier for you when it comes time to commit to the Git repo, put this gitignore file inside the maven folder. (If you’re using SVN, it’s literally the same, just replace “git” with “svn”)

*
!.gitignore

This tells Git to ignore everything in this folder except for the ignore file.  We do this so that the folder itself gets checked in, but everything inside of it does not.

Dependency Management among Assets
A nice feature of maven, is that a particular binary can have a dependency on another binary, and Maven will auto handle all the dependency. In layman’s terms, AssetA needs AssetB to function.  When we Maven-ify AssetA, we declare a dependency on AssetB.  When we tell Maven to fetch AssetA, Maven sees that AssetA needs AssetB, so it automatically fetches it as well.  Maven will also see if AssetB has any dependencies and see if those dependencies have depencies and etc etc etc…..

When we maven-ify an asset, we declare it’s dependency the same way we do in our main project. Nothing different there.  A good use case of dependencies is large assets that you might only want individual chunks of.

Let’s say that Unity-Chan had a part that was an alternate model. We could maven-ify the main asset as
groupId: unity.com.unity-chan.unity-chan
artifactId: core
version: 1.1

and it’s the second part as
groupId:unity.com.unity-chan.unity-chan
artifactId:alternate
version: 1.1

Then if you only needed the base Unity Chan model, you would only declare a dependency on the core artifact. However, if you needed the alternate model, you would use the alternate artifact, and Maven would pull the core down as well. As far as Maven best practices go, you list any dependency that you are directly using, and let Maven handle any non listed dependencies.  In the above example, if you were using both the core model and the alternate model, you would list both (even though it would work with just the alternate listed)

You would need to modify the includes and excludes tags in assembly.xml accordingly to only grab the folders you wanted.

Ending Notes
As I mentioned above, Maven is designed for Java not Unity. So if any of this feels hacky, that’s because it is.

Assets with multiple root folders or ones that need to be in the root Assets folder (i.e. Plugins, Editor, etc) can take some extra work.  For the former, I usually just dice it up into separate artifacts for each folder.  For the latter, I haven’t found a good work around other than just checking that specific folder into our version control.

You could definitely enhance this workflow and automate more processes using something like Gradle. However, my worry is:

In the end, this has worked pretty well for our team. I’m not saying this is the best way to do things, but it’s a tool we use that works for us.

Categories: Dev Blog,

Leave a comment