Lucas Meijer

Okay, well maybe it’s not a manual, but at least it’s a description that I could understand about what svn_load_dirs does, why it is useful, when you want to use it etc. Maybe it’s useful for someone, at least I hope it’s a reference for myself in the future, as I will no doubt quickly forget what I’m about to write.

Many people use vendor branches in their subversion repository. When you depend on some external project, and you want to store its sourcecode in your own repository, you generally just commit it somewhere in your repository, and you call that a vendor branch. Instead of directly committing into your “main tree”, you generally place it somewhere outside. Let’s say /External/Mono. I’ll use mono as an example, because that’s the external dependency I actually deal with frequently.

It turns out it’s comfortable to actually place it in /External/Mono/current. that way when you put a new drop of the mono sourcecode in that place in the repo, you can make a tag at /External/Mono/mono2.6-beta3. Tags are cheap anyway, and it gives you an idea of what the hell you were doing.

Now, in most cases you’ll actually want to modify the vendor’s sourcecode. (If not, there’s not that much point in versioning it). This is where it gets a bit tricky. What I do is actually also have a copy of the mono tree in the main svn tree. This copy is initially made using svn copy. Whenever /External/Mono/current gets updated with a new drop, I svn merge those changes into the actual “real tree”. In order for this merging to work, you actually need to make sure that the svn:mergeinfo property gets properly committed, after merging. (if in the ‘real tree’, you placed the copy at myrealtree/Sources/Mono you need to have the svn:mergeinfo on that mono folder).

So now when you want to make modifications, you just change whatever you want in myrealtree/Sources/Mono. The actual vendor branch at /External/Mono/current should always contain a clean version of the vendors sourcecode.

Now, let’s say we want to upgrade our version of Mono. That means we want /External/Mono/current to now contain updated code. This is where the tricky part starts. What are our options of making this happen? We could just make a checkout of /External/Mono/current, and just copy over the new sourcecode, and commit. This has a problem however:

What happens to deleted files? files that used to be there, but now no longer should be. If you just copy over the new sourcecode, those files will still be there from the old sourcecode.

What happens with renames? Ideally, if the mono sourcecode renamed GZipCompression.cs to GZip.cs, ideally you want that showing up as a rename in our own repo, so our own changes to that file will get carried over.

In practice, I’ve found I can deal with the renames, but the deletes are really quite a problem.

This is where svn_load_dirs.py comes into play. It’s job is to fix this problem. You tell it which repo url you want to be changed, and you tell it which local folder contains the new sourcecode that it should be changed to. You can also ask it to make a tag for you automatically. Let’s take a look at the commandline:

svn_load_dirs.pl -t trunk-r141102 -wc vendorbranchatunity svn://mycompany.com/svn/External/Mono current monoprojecttrunk

this says a few things: the main “vendor branch area” is svn://mycompany.com/svn/External/Mono
please note that this is the parent folder, of /External/Mono/current. we give it the parent folder, because we are specifying two folders that are actually relative to this. the folder where the actual new drop should go, (current), and the folder where the tag should go. (trunk-r141102). So after this process, the svn repo will look like:

/External/Mono/current (with new stuff)
/External/Mono/trunk-r141002 (same stuff, but tagged so it’s easy to find later)
/External/Mono/trunk-r120000 (you might have an older tag from a previous run)

Okay, about the other arguments. the last one is “monoprojecttrunk”. This is a local directory containing the actual sourcedrop that should be placed into ‘current’. In most svn_load_dirs stuff I found online, people suggest this should be an svnexport of wherever this code originated from. For me it has worked just fine to have this be an actual working copy of where the code came from. My code comes from the mono project’s svn. That way it saves me an svn export, in what already is a lenghtier process than you’d want. I do make a few modifications to the local mono checkout though, as it contains a lot of heavy stuff that I don’t use. moon/test for instance contains a lot of testmedia that I don’t care about, and there’s some more cruft (cruft from my pov, useful stuff for others ) that I cut at this point.

These are all the essential arguments that you need to do the real work. There’s one argument that is merely an optimization. it’s the “-wc vendorbranchatunity” one. If you don’t provide this, the script will just make a checkout of the svn url you gave it. Since in practice it takes at least a few tries to get the script to do what you want, making those checkouts all the time takes a lot of time. So I always just make a checkout myself, and tell the script to use that. I make a backup of that checkout, so that when the script blows up on me, I just restore my backup, fix the reason why it broke, and try again. The actual svn url that should be in the checkout is the ‘current’ directory. so not the base url that you pass on the commandline, but the base url + current. (there’s no rule that says you have to call it current, it’s just what I use, and I think the svn book also uses it as an example).

Okay, so we got the whole commandline arguments thing figured out, let’s actually run this thing.
The script will first do some sanity checking, make sure what you say is an svn url actually works, yadayada. Then it will analyze the files currently in “current”, and check them against the files in “monoprojecttrunk”. it makes a list of files that should be added, and files that should be deleted.
It then presents both of these lists to you. You can inspect them for a sanity check. It then asks you if you want to mark any combination of those as a rename. So you can go in and say “Yeah, actually that GZipCompression.cs that you are deleting, and that GZip.cs that you are adding, that is actually a rename”. that way, it will be applied to your svn repo like a rename, which then makes it easier to merge that change into your real tree, especially if you happened to have modifications to that file.

Usually I don’t bother to actually go in and tell it about the renames, because I have checked that the files that I have actually modified are not renamed, so then I really don’t care that much.

Okay, so once you go that out of the way, the script is going to svn-add all over the place, and svn-rm all over the place, and then it’s going to try to commit it all, commit the tag, and then be done.

Unless…

unless some of the changes are actually renames in casing only. It turns out that the mono folks have been cleaning up their shop a bit, and fixed some casing issues. stuff like renaming ChangeLog to Changelog. It turns out that svn_load_dirs is not capable of dealing with these cases gracefully. It will give you this error:

svn: Commit failed (details follow):
svn: ‘/Users/lucas/mono/vendorbranchatunity/mcs/class/Mono.Simd/ChangeLog’ is scheduled for addition, but is missing

When you get this error, the casing problem is probably what’s hitting you. The script will just bail out, and you need to fix the problem manually. I fix this by going into the checkout of ‘current’, and just deleting the file there. commit the delete, and then start the script from the beginning again. (This is where the -wc optimization comes in handy).

When this has succeeded, you should be happy. And then move on to the following step:

merge the changes that just got applied to ‘current’, to the version in your realtree, which actually contains your modifications. I like to use tortoisesvn for this, as it’s recently improved merge gui is actually quite decent, and by this point I’ve usually had my share of commandline argument struggling for the day. so, merge the changed revision into your real tree. (and make sure that when you commit that, you also commit the realtree/Sources/Mono folder itself, because it’s svn:mergeinfo will have changed, and if you do it right, it means the next merge is going to go a lot easier than when you decide to forget about this for now.

Hey,

I’m happy to say that I have joined Unity.
I haven’t really decided where my new blogposts will go, but for now I’m keeping this blog up for stuff that is not suited for the unity blog itself, because it’s either not totally about unity, or maybe too detailed.

Bye, Lucas

sounds familiar?

You’ve carefully placed that one light in your scene. Or tweaked your physics settings untill they were just right. Yes that’s it, just a bit more to the right, yes, yes.. yes, perfect!.

And then you realize that Unity is still playing, so your changes will be discarded when you stop playing, and you have to do the whole thing over again.

Welcome to the club, you’re not alone.

What can you do about it?

In Unity2.5, it’s already harder to make the mistake, as the entire IDE gets tinted dark when your game is playing. So think dark->dont-spend-hours-tweaking.

Okay, so in case you forgot, there’s still a way out. There’s a trick to preserve your changes.
While the game is running, select the gameobjects that contain your tweaks, and Copy them. (Apple-C / Control-C) Stop the game. Paste them. Remove the old gameobjects. (Or use the pasted gameobjects to only copy some values from into the old one).

If you’ve already clicked stop, and only then realized your changes are gone now, then… I’m sorry, they’re really gone. If you realize you’re in trouble before you click stop, you can still do the copy/paste trick.

As I was looking at the web traffic for this blog, I noticed that the post on Visual Studio integration for Unity was getting most traffic by far. Too bad the post was from a 2.1-mac era, and it actually doesn’t work anymore for 2.5-windows, which I suspect most people are looking for these days.

I updated the integratoin to work for Unity 2.5 on Windows, Visual Studio Express, Visual Studio pro, and MonoDevelop. I also created some item templates for Visual Studio, more on that below.

It’s amazing how much convincing it takes to get people to try out Visual Studio for development. I’ve heard every excuse imaginable so far. If you take anything away from this blog, let it be “maybe I should try that visual studio Lucas keeps nagging about”. Or try MonoDevelop. Or SharpDevelop. Anything but Unitron or UniScite or Notepad.

Did I mention Visual Studio Express is free, and does everything you need?

To use this, follow these steps:

• Download this .cs file, and place it in your project, in a folder called “Editor”
• Notice that you now have a Tools menu item, containing a SyncVisualStudioSolution item
• Click it
• Open the .sln file you’ll find in your project root folder

Every time you click it, the .sln and .csproj files get recreated. In daily usage you won’t do this very often, as a nice workflow is just that you create new scripts in visual studio itself, instead of in Unity. As long as you create scripts in visual studio, Unity will automatically pick up those scripts, and your Unity project and your VisualStudio solution will be in sync. If one day you decide to create a script in Unity, just click the SyncVisualStudioSolution button, and your solution will be recreated, with all scripts and shaders that live in your Unity project.

The created project directly references the UnityEngine.dll and UnityEditor.dll in your Unity installation, nothing you need to do there. For the best experience, download these xml documentation files, and place them in your C:\Program Files\Unity\Editor\Data\lib folder. They will get you the Unity documentation in tooltips as you are using methods in VisualStudio. Very neat. (Thanks to Benoit Fouletier for his regex-fu)

Are there no downsides?

Not many, but here’s one. Because Visual Studio requires the folders contained in its projects to be in a directory under the .csproj file in order for create->new code file to create files in those folders, instead of in a “fake” folder, I chose to place the .sln and .csproj files in the root of your Unity project.
Unfortunately this means a bit of clutter: sln, csproj, suo, obj, _Resharper, are some of the files/folders you might see laying around in your Unity project folder now. (Not in Assets/ mind you, in the parent folder of Assets). There is nothing inherently bad about this, except that doesn’t look very tidy.

Item Templates

Which results in this file:

Note how the name I chose in the first dialog is inserted in several places in the actual editorscript.

To get these, download this file, and extract the two zipfiles that it contains into your equivalent of
C:\Documents and Settings\Administrator\My Documents\Visual Studio 2008\Templates\ItemTemplates\Visual C#

I couldn’t figure out how to put these item templates in the “Code” subsection. Bonus points for whoever figures it out, and double bonus points for who figures out how I can specify a starting point point for the cursor, or an initial range of characters that should be selected.

Prefer MonoDevelop?

That’s cool. MonoDevelop reads .sln and .csproj just fine. So if you’re on a mac, and want to use MonoDevelop (make sure to use the newly released 2.0, it has gotten a lot better), go nuts. Just open the .sln file and you should be good to go. In theory

I hope people find all of these useful. Drop a line in the comments if you feel like it.

Hey,

Just got back from GDC 2009. A lot of stuff happened, but this post is about the release of Director 11.5

I’m happy to see that after so many years (I’m counting 8), there is finally a director release that has some actual new runtime features:

• New audio engine. (5.1 + Filters)
• ByteArray support. (One of many reasons I’m using Unity on my main project now is that up untill now it was impossible in shockwave to have a binary protocol with a server that is able to send the 0 byte)
• Native rendering of H264 and FLV/F4V
• Sketchup import

Congratulations to Allen, Rasjnish, Krishan and the rest of the Director team for getting it out the door.

It’s out. It’s live.

The inevitable has happened.

Unity2.5 is out, and it runs on windows.

Here is my personal favorite list of changes Read the rest of this entry »

Remember the message window that Director has? Very useful if you just want to quickly try out something. Do it while your game is running or not running, it’s all good.
It’s one of the few Director features that I’ve missed in Unity for a while. When Miguel de Icaza announced an interactive c# program, I was very excited, as this opened the door to a message window in Unity.

I’m not completely done yet, but check out this preview of an in-unity c# “message window”.

(not embedding as the swf has huge dimensions)
http://lucasmeijer.com/files/messagewindow.swf

A few notes:

• This works at runtime, in your webplayer too. (at a hefty 3mb pricetag)
• This should work in Unity 2.1 as well.
• Careful readers will note the usage of the ‘var’ keyword, which is not supported by the version of Mono that Unity embeds. This is because this tool embeds its own c# compiler, and I chose to use a much more recent version of the mono c# compiler than what Unity ships with. Don’t look at this video and think Unity2.5 will ship with an upgraded Mono.
• Still have lots of stuff to add. Tab completion, Intellisense, shift-return executing multiple lines, being able to drag gameobjects in, and have them be replaced by GameObject.Find(”thename”);
• Miguel+friends did all the real work, I just put a few pieces together.

All in all, it’s great fun to work on some tech purely for fun again.

I’ll probably release this as an open source package when it doesn’t require tiptoeing around all its infancy problems. Maybe I’ll talk the unity guys into including it in the next version.

I think this shows again what great move it was for Unity to embed Mono. Mono rocks.

This is my “Things not to forget when building mono on windows” list. Hopefully it will be of use to somebody else too.

• Remember that this always takes a lot longer than you like.
• Read this first http://www.mono-project.com/Compiling_Mono_on_Windows.
• use this:

  cd mono
  make get-monolite-latest

  to get a mcs compiler installed that you will need to compile some mono components.

• Do not forget to use the make linked to in the mono compiling on windows guide.
• When the guide says “. /tmp/build_deps/env.sh” that first dot is _not_ to be forgotten.
  You can also use “source /tmp/build_deps/env.sh”. do a sanitycheck with echo $PKG_CONFIG_PATH to see if your paths are really properly set.
• Remember that pkg-config is a program that gets run by autogen.sh. Autogen.sh basically asks pkg-config “where is library ‘glib’, and what compiler and linkerflags should I add to use it”. try to see if your pkg-config works first. just write “pkg-config –libs glib-2.0″. If that doesn’t work, fix that first.
• If your pkg-config first worked, but later didn’t work anymore, it is likely that you modified your $PATH environment variable to contain a path to a working mono setup, to get a working mcs for your mono build. If you placed this new path in your $PATH before the rest, it means you’ll now accidentlally use /your/mono/bin/pkg-config, instead of /usr/bin/pkg-config. That is likely a too old version, and actually fails.
• Do not forget that the relocate-dependencies script linked to in the guideline actually hardcodes a C:\cygwin path. Mine is at H:\cygwin.
• If you get freezes or errors on something that says [MDOC] blabla.. try appending –with-moonlight=no to your autogen.sh command.
• If that doesn’t work (as it didn’t for me), manually edit mcs/Makefile, and remove all references to “docs”.
• Since we patch mono with monoco, don’t forget to apply the monoco patch. If the monoco patch
  doesn’t cleanly apply, it’s usually because of conflicts in mono/mono/mini/Makefile.am, which can easily be resolved manually.
• Remember to remind linux users that most opensource windows software you can just download, doubleclick the .sln, and have it work. No rocket science needed.
• Remember to run around like crazy, arms waving, asking yourself why linux projects don’t just ship with their dependencies. (Except for MythTV that just embeds a forked copy of ffmpeg that they update from time to time. Hurray for common sense)
• Remember to find a copy of the autoconf manual, and burn it.
• Remember to tune into http://linuxhaters.blogspot.com/

edit:typo

Making sure you have 0 warnings is good advice. Unfortunately, I’m not perfect, and in one of my current projects, I have a pretty big list of warnings that I know are not important.
Today, I got bitten by not following the 0 warnings advice. I searched for a bug for over an hour. Let’s see if you can spot the mistake:

if (m_holdingMessages);
{
    Log.Msg("Holding messages in queue");
    msgQueue.Enqueue(msg);
    return true;
}

The if statement always executed. m_holdingMessages should have been false, and I was pretty sure it was false, but the code executed anyway. Being the witty reader of this blog that you are, naturally you immediately spotted the erronous semi-column after the if statement. It got there accidentally, but funny enough the resulting code is valid c#. It’s silly c#, but valid.

Naturally, Visual Studio, (and Unity) give warnings when compiling this. They say: “Possibly mistaken empty statement”. Dead on.

But I missed the warning, because it drowned in all those warnings I had been ignoring all that time.
Had I been a good boy, I would have made sure I had no warnings at all, and this one waring would have gotten my attention, and I could have spent my morning drinking coffees instead of looking for this silly bug.

I frequently get emails along the lines of “hey, what do you think is a best for my project. director or unity”. The answer to that question totally depens on what kind of project you’re making. Let’s do a quick featurelist:

Unity3D

• superb 3D engine
• crap 2D support. No layered psd import, no 2d layout functionality
• wii support
• iPhone support
• webplayer support. plugin penetration +-0%
• webplayer with very clean install. Claimed by Unity to have a 50% succesful install rate
• standalone support (pc/mac)
• blazing fast scripting platform
• okayish documentation. not great, not bad

Director

• mediocre 3D engine
• crap 2d import pipeline, but great photoshop support trough PhotoCaster Xtra. Okay 2d layout capabilities
• flash integration
• webplayer support. My guess is plugin penetration is around 50% for mature markets. (*)
• webplayer with opt out google toolbar, and opt out Symantec anti virus included with installer. (**)
• standalone support (pc/mac)
• slow scripting languages
• horribly outdated and incomplete documentation

So who wins? It depends on what you want to do. Here’s my list:

• offline 3D application
  Unity wins bigtime. Better 3d engine, no serious downsides.
• online 3D application
  Unity wins, but a closer call. Better 3d engine. Penetration disadvantage (which I’ll get to in a sec)
• offline 2D application
  Director wins. Flash also already covers this area quite well, but I can see there is a niche inside the offline 2d arena where using Director makes sense.
• online 2D application
  Director wins. But Flash beats Director here. Flash has 99% penetration rate, and can do pretty much anything Director can do 2d wise..
• if you want to bring anything to iPhone
  Unity wins. Director is no option.
• if you want to bring anything to Wii
  Unity wins. Director is no option.

I think the only somewhat controversial item on this list is Unity winning the online 3d game category. “0% penetration rate, get lost!” one might think. But if we do some math, the difference isn’t quite as big as it feels (***):

Potential audience for Unity Game = Installbase + (100% – installbase) * successful_player_installrate.
Potential audience for Shockwave game = Installbase + (100% – installbase) * successful_player_installrate.

Not all the variables here are public. If we believe the numbers Adobe claims and Unity claims, we can fill in some:

UnityAudience = 0% + (100% – 0%) * 60% = 60%
ShockwaveAudience = 50% + (100% – 50%) * shockwave_install_succesrate = ?

I wonder if Adobe would be willing to share the successful install rate number with us. With the payloads, and the required browser restart, I would guess them being at 40%. I could be high, I could be low. If anybody knows the right answer, please let me know. Let’s fill in 40%.

ShockwaveAudience = 50% + 50% * 40% = 70%

So it’s a 60% vs 70% situation. That looks a lot different than 0% vs 50%.

Is there any market left for Director to compete in? In my list above I have it winning only in Offline 2D. A market that is already fairly adequately covered by Flash.

Naturally feature sets alone is not enough to base your decision on. If you have 3 experienced Director devs, and no experienced Unity devs, that would make the balance move more towards Director.
If you already have a few Unity devs, Unity would come out more favorably. I haven’t included price in the comparison yet, as my reference frame is mostly projects that have a scope where a $1000 license fee is not a big enough deal to base a tool decision on.

Am I being too harsh for Director here? I’d love to read your comments if you feel Director makes sense to use in a certain area that I don’t, I’d love to hear your arguments. Here’s some I can imagine coming up:

• director is extendable trough xtras
  So is Unity.
• but director also has this extendability in its webplayer
  Unity also has this, as long as you confine your extendability to a .net assembly. Also, the extendability in the shockwaveplayer brings up a very scary verisign dialog that will scare away even more users. Every sane dev I know tries to not use non-adobe xtras in online shockwave applications.
• this 60% vs 70% difference matters!
  It does. However, Unity’s much more efficient workflow easily compensates that difference imho. You’ll spend a lot less money developing, leaving you more money to make your content better.

Do you know more?

(*) Adobe claims a 58.2% penetration rate for shockwave 7 and higher. It provides no data on Shockwave 10 or 11. If you want to do 3d you at least require shockwave 8.5. So actual penetration rate depends on the shockwave version you target. 50% is my guess for shockwave 10 penetration.
imho, it’s a bit silly that adobe measures shockwave7+ penetration rate, as shockwave7 is useless, making the stat useless. At least shockwave8.5+ should be measured. And ideally a better overview of penetration by version would be provided. Emerging markets were last measured at 34% for shockwave7+

(**) The shockwave installer has a opt out google toolbar and a opt out symantec antivirus payload. However you always get “offered” (I would say tricked into) only one thing in a single instal.

(***) example used from a David Helgason blogpost.