Behind the scenes, shots and, er… files
/ by Matt
I’ve had a few questions about how we’re organising our work here, so I thought I might give a little run-down on it. It’s been interesting figuring out ways that we can collaborate together easily, while also being able to re-use models, sets and characters.
Blender offers a pretty wide range of options for managing assets with its library linking system, and we’re using it a lot. As well as appending things straight into a local file, you can link any blender datablocks like objects, object data, actions, or entire sets from an external library .blend file. Blender also allows multiple Scenes in each file, which can be completely independent, or have various datablocks linked between them. The trick for us was in deciding what features would suit us best (and what should be coded :).
As mentioned in this post from September, we’re using Subversion on a server in the studio to manage the files and revisions. Among other nice features like being able to roll back to past versions of files, one of the greatest advantages for us that we weren’t really anticipating, is that with the subversion commit and update workflow, everyone keeps a copy of the project’s entire folder structure synced locally on their workstation’s hard disk. This means that we can use relative paths in Blender (eg. //../lib/machine/blah/file.blend ) to reference the same .blend files, textures, sounds, etc, no matter what sort of folder structure the subversion tree is situated underneath, and without having to work off a (comparatively slow) network share.
After some discussion, we settled on a system: We have a production folder, with a lib folder inside that containing all our libraries that we link, with subfolders for set and prop .blend files, subfolders for emo and proog, a subfolder for textures, one for audio, and so on. Also underneath production is a folder for each scene, with a number and short descriptive name. Inside the folders for each scene, we have separate .blend files for each shot, named as SCENENUMBER_SHOTNUMBER.blend . See the screenshot on the left for the production folder structure (with names blurred to prevent spoilers ;).
We decided on this for a few reasons. While making the animatic, we were generally working on a scene each, using single .blend files per scene, with a python script to change camera angles for each shot. This was nice and fast and flexible at the time, but it had drawbacks such as not being renderfarm safe. We could have also done one scene per .blend file, with separate Blender Scenes (that we call ‘Beans’ to differentiate between those and those in the movie) for each shot, but we decided against that too, since it’s far easier to collaborate with one .blend file per shot. This allows us to easily tweak things such as lighting setups per shot, but most importantly, it splits things up to allow us to collaborate. Unlike the animatic, we’re dividing up the shots between us as we progress, and need separate .blend files to work on and commit updates to SVN individually.
Blender has a feature where you can link Blender Scenes into another Scene as a Set, meaning that the objects show up ghosted and un-selectable and un-modifiable. This may be convenient for some purposes, but we’ve decided not to use it, as it doesn’t give up enough flexibility to make little adjustments to the shots, to tweak compositions, or cut away parts that don’t need to be there. Instead, we’re using a great new feature that Ton coded for us in the Orange development branch of Blender: Groups.
Groups are abstract names, that you can assign Blender Objects to. You can add and remove objects to and from groups at will, and you can also link the Group from another file, just as you would an Object. Grouped objects display with green wireframes in the 3D View. The advantage of groups is that since you’re only linking the Group name, you can make any adjustments in the library file, adding or removing objects, editing materials, whatever, and it all gets linked through to the other files dynamically. To instantiate these groups within the scene, you can use the ‘dupligroup’ feature of objects, which creates an instance of that group on that object (generally an empty). There’s even a nice little menu in the ‘Add’ menu, containing the available groups, which will add an empty with the specified group attached to it. We’re using these a lot for set and prop items, so we can have the convenience of being able to move things around, but without keeping local data.
It gets even better though. If the group contains an Armature, you can also assign pre-made actions to that group’s instance in the NLA, even if it’s just being instanced on an Empty. Type in the name of the Armature in the group that you want to assign the actions to in the ‘target’ field of a strip’s properties in the NLA. Since you’re really only dealing with an Empty, you can’t actually pose the armature, so unfortunately it’s not useful for main characters, but it’s still extremely useful for lots of secondary characters that we can assign little actions to, to give life to the scene. You’ll see what I’m talking about when you watch the movie ;).
« New Mini Production report | Deadlines! »
Yay first post
Cool I can’t wait
For the movie to be out!
second!
you guys rule.
.b
third! no *you* rule!
Got to agree with Bassam here.
´
The best part it seems about having all files locally synced to computer is that there’s a bunch of backups, something to the effect of a massive (if sometimes asynchronous) RAID array.
Of course nothing 100% to be original in that situation but it’s better than everyone working off a central source somewhere and only uploading when they need to.
Just out of curiosity do you guys do any off site backups?
yes, using a hard disk, Ton, a bike and his apartment.
~Toni
-YOU GUYS RULE!…….continious…..YOU GUYS RULE……continious!………Amazing!…..Great!….YOU GUYS RULE!……continious……!
What a Development!
Congratulations and all the Best!
i am always late
That backup strategy is seriously flawed, what if Ton gets hit by a tram? I suggest that the entire orange team takes a copy of the harddisk, get their bikes and each ride in a different direction, to avoid being hit by a meteor.
Roel
I was just wondering… do you use any Windows workstations, or are all you workstations running ‘open source’ software and operating systems?
Mac OS X isn’t really ‘open source’, only their Darwin and X11 window core.
hehe, you guys need to bathe – you have fleas!
/Nathan
you thought you were late!…anyways, keep up the good work
/Chris
MacOS X(tm) is already more ‘open source’ than Windows(tm) :)
(Also, there is a bit more open softwares in MacOS X(tm) than Darwin and X11 Core)
—–
Great use of SVN, just by the way, will you merge Blender CVS repository to SVN in the future?
Big up mens, you are the pioneers.
Have you ever herd the tall tale of
The rooster that landed on the moon.
Well if you have let me be the first to
Say “Danger keep out of Gordon may explode”
And lick “Danger do not eat to much Peanut Butter
May EXPLODE!!!!!” If you happen to be walking
Down the street and see a can run to your house and put
The can on top of your printer and print out a can of wall
Paper!!!!!!
is that why my Peanut Butter is always in a can?
That was the weirdest comment I’ve ever read in my entire life…
Uh..
Keep up the great work! Woooo!!! …Wooooooooooo!
That’s right, woo!
Matt: could you give more explanation about Groups, i didn’t get it.
thanks
No Ok, i got it!! :)
You got what? :s
Matt makes nice long explaining post for community.Why he bothers I don’t know. About 14 out of 18 responses worthless dribble…
So this is stuff that raises Blender profile in cg world?
Come visit Orange and see best of Blender.
Enlightening comments for visitors like ‘#### I am always late’ or Timothy idiot first post and nonsensical paragraph…you on drugs or just nothing between ears? If you the same guy who runs Elysiun no wonder poor moderation.
You know I can’t be bothered posting in various Blender places now and this very good demo of exactly why.
Absolutely no idea of how to behave for public.
Out of curiosity (mostly because I’ve never taken the opportunity to play with it), how does the library linking system differ from the straight-up SHIFT+F1 Append?
I noticed that the Outliner has the option to view Library data blocks, but it seems Blender (both 2.40 and a CVS checkout from today) likes to crash if you press this button after doing an Append.
Any helpful input?
Yes guys, please endeavour to appear at least somewhat sane when posting here :)
Felix: Our equipment is documented in this blog post from September here (hrm, I really should look at categories and search for this blog…) /blog/equipping-the-studio
Fweeb: Linking is different in that it doesn’t import a local object, it keeps a reference to an external .blend file. These references are upated when opening the container file. So you can link an object into another scene in another file, then later on go back and edit the original object, then next time you open up that original scene, it will have automatically sucked the updated, modified object back in.
Which option do you mean to view library datablocks? I’m not really sure what you mean.
Cheers
Rubbleman,
the stupidity was us fooling around ourselves, as a reaction to the funny ‘1st post’ first post – a result of working too late, but i guess this is no time to censor the blog anymore.
Felix, so no win32 here – in principle we could use it, ’cause open source apps such as Subversion, Blender etc. usually work ok on windows too, but .. i guess we just dont have a reason to use it, posix systems such as Linux and Darwin are of course more compatible with each other too so we dont have to worry about e.g. filesystem level differences that we might encounter on win32. Back home at the Kyperjokki studio we actually have all Linux, Mac and Win32 running the same systems, and with the current cross-platform techs such as SDL, OpenGL and Python things are pretty smooth. And in fact right now we are giving a workshop downstairs here at Montevideo, some of the machines (from the house) running win32.
~Toni
The reason you’re so late is because everyone commenting early is hooked up to the server and gets i.m.’d whenever anyone coughs. The marvel of modern technology. Just try getting that kind of data over the teleprompter. :) or the telegraph.
I kinda feel sorry for Timothy. Click his link to see who’s behind the name. He reminds me a lot of my 9-year-old. It’s hard for a smart young man to know when to keep his mouth shut.
Timothy: I know you’ve done a lot of cool thing with blender, and you appear to be extremely smart for your age. If you want to be accepted, please learn to stay on topic. The purpose of these forums is for us all to discuss the project and *learn* from it, not just to let people know that we’re out here.
Rubbleman….I dunno if you are joking or for real but I don’t mind the tone of the other posts up here. It’s the internet, it is what it is. Personally I’m enjoying the heck out of it.
Matt: In the OOPS view of the Outliner… it’s the last button on the right with the letters “Li” on it. The tooltip says, “Displays Library datablocks”. And I suppose I wasn’t entirely clear… I should’ve said, is the process for linking to a blend different from appending? If so, how is it done?
Yeah… basicly it was a convoluted way of saying “how do you make links?” I found it in the documentation (for those following along, SHIFT+F1 lets you Append or Link… it just depends on whether you select “Append” or “Link” at the bottom of the file select window). Of course the behavior in OOPS is still odd.
Little Timothy is cool! Don’t give him a hard time (even though that post was a bit strange). I checked out his entire website and he seems to be a rather smart kid who loves Blender and actually has some real skills (check out his animations for his Dad)! Who knows, in 10 years he might become an integral part of Blender development.
Welcome to the Blender community Timothy! Just keep your posts in line with the topic being discussed and you’ll do fine.
Actually, all this talk of shots makes me curious:
You said you split the shots up into files… by this you mean that every camera change is a different .blend? If so, exactly how much of the scene is linked from outside files? You say you can adjust light by-scene; does this mean that you’re creating lights and sets in another file, linking them, and then playing with them by-shot, or are you adding new lights in the shot files?
Anyway, I hope a small bit on how you guys went about dividing up things and how you linked everything is on the DVD, explained such that even someone like me can understand. ^-^;
Hi,
First : felicitation, good work !
The Orange project is a very interesting project.
I wait impatiently the DVD …
But I have one question (Excuse me to post this message here but I didn’t found other place or e-mail)
Do you plan to do a translation of the Movie/Making off in other languages (french for example ;-) ?
Just Simple subtitle would be very good.
I know translation is a long work but i think it could be very important for a large diffusion of this project.
Maybe I have post this message too late…
I hope no :)
Thank you very much,
Fowlalgorn_gui
And what about verse ? Did you tried to use it for some tasks ?
O.K. Kernon so all I do is
Not change Subject that sounds
Ease.
Damm interesting use of groups, thanks guys, I will be using linked groups a lot :-)
Yes Trevor I serious about it, maybe internet is place where freestyle and ignorant manner is acceptable to you but still lot of people have ideas that intelligent conversation is possible.Blender also gets external money and sponsors to help make things like this possible.These persons don’t want to connect business name with embarrassing conduct of rabble.
Orange is also showcase of program capability and talent to establish credibility in cg world.
Not good look for important persons,professionals and public to come here and read nonsense from community.Imagine Ton goes to next siggraph and people think outfit is poor joke based on what they see here.
Matt already is very busy and we are fortunate he makes this well explained post for us.Wouldn’t hurt at all to have replies reflect the topic he made. Is respectful and fair to him too.
If Blender was in position where rely on sales for existance you don’t make this type of mistake. I am sure persons who visit here know how to conduct themselves well if they concentrate a bit realising they are not just having rough house private conversation with the guys but also representing Blender indirectly.
I see you have business yourself. I think you capable of understanding what I say quite well.
You know, I’ve been following this blog for ages, checking it every other day at least… and have never said anything. But, in light of the inanity of certain posts, I suppose that a foolish and purposeless comment would fit in as well as any other…
And all I have to say, after this unfortunate preamble… is that I have the greatest respect for everyone working on this project. Your work is one of beauty, not merely in principal, but I imagine also in practice where the finished product is concerned.
My kudos to all of you for not only providing a concrete workflow to inspire the improvement of some of the most amazing opensource software in the world, but also for using it to achieve a creative vision of beauty. You are all an inspiration.
It ‘s a good work.
I’m waiting to see when it finish.
Hey Orange people!:)
I just whant to say, that you work is GREAT! I’m so glad you started this project! keep up the good work:)!!!
I was wondering… Through these months orange team learned a lot, and now they became a productive group in the field of animation with blender. Isn’t sad that all this know-how will be lost with the end of the project? I know is a matter of costs, but would be great if they should become a stable production team for future works. (with the addition of endre barath as artist…i love his works)
Epsilorn Wrote:with the addition of endre barath as artist…i love his works)!
also Enrico Valenza and VenomGfx and the Plumiferos Team!!
and some others!!!?
Regardless of whether the Orange team stays together or breaks up after this project is done, they have without a doubt inspired others to produce the same quality of work using open source software. It’s a win for open source, and it’s a win for the blender team.
The press that the orange team has received, and will continue to receive for their efforts has brought blender to the limelight, and pushed development over the edge – and we, the community have come out on top – can’t wait for the premiere!
Cheers,
ibw
Rubbleman…
I´m sure your opinion is in the best interest and
best wishes for the Orange team & Blender foundation
but consider this:
This is a blog – a way for the Orange team to
communicate it´s efforts and progress to everyone
who is interested, this also includes the community.
And who is the Blender community? It is a lot of different
people all over the world, all ages, all cultures and
very different from each other – with ONE common interest,
namely – Blender!
I agree that certain things should be moderated such
as spam. What do I consider spam? I consider ads for casios
viagra and spam, not drivel from some of the community members.
I understand your worry about keeping the blogs clean
and on “topic”, but it is a community. Sometimes people
in the community feel like they are belonging to a group,
sometimes the group can be felt like a family. As you can
see the Orangers probably feel this way too as they often
humanly enough – themselves participate in the simple
jokes amongst the community members.
The frontpage of Orange.blender.org and the various pages
are neat and clean and well worth the sponsors names.
This is a community response page! Even if it´s just
to say “hi!” or “Basse..you´re the greatest!” it would
be all okay…because its the community – take it for what
it is.
If this was a Forum – I´d agree with you in a second!
But it´s not. It´s just fans, users, visitors and the rest
interacting with each other, kind of like a family.
I´ve never met anyone of these in person, but I´ve seen
many of these peoples work, many of them have seen my work
and others have seen each others work. It´s “okay” for us
to blabber about a little bit in this “freespace” of user
interactivity. At least I think so ;)
Long live Blender!!
And thank you Orange team, for making it shine.
Actually, I guess blender is also making the Orange team shine!
Have you seen what other folks have to deal with. I just saw the demo videos of Pelting for UVs. Its so funny, and it leaves little spikey things all around the border. Live LSCM is so much more efficient! This may be anathema, but could live LSCM be implemented in a MEL script. I’m sure Ton could do it in 20 minutes!! He is so awsome. I hope he still has time to code when Orange is over.
Anyway, keep up the excellent, jaw droppingly awesome work!!
Yeah ok Mr Jongle I give up. You grunt here if thats your style.
You have grunt party with your family if you like while at it.
Old rubbleman whos time is coming soon had thoughts of leaving generous money to Blender for various purposes but no more. Enough community persons demonstrate will not be respectful or deserving of it.I do better good with funds somewhere else.bye
Rubbleman,
Funny reaction. Imho, that shows me more immaturity than the audience you just mocked earlier. You obviously wanted to engage in an intelligent discussion, you do NOT do this by mocking people, you do this by setting an example.
In regards to whom´s to blame for your lack of wanting to donate to the Orange team or Blender is entirely up to you – not me. I have donated to the project myself in the beginning, but that´s no mark why you should…that is entirely up to you, not me.
The classic response you just gave:
“Old rubbleman whos time is coming soon had thoughts of leaving generous money to Blender for various purposes but no more. ”
Was exactly like a little kid that didnt get his way, just like some customers behave in a shop if they don´t get sucked up to…by leaving while voicing their “last-words-loudly” about the “missed-opportunity”. Hah, you had no intention of donating anything.
Hey guys, I don’t think this sort of conversation is very nice, and it’s definitely not on topic. This blog is not only our way of communicating, but I hope it can remain as a useful knowledge resource with hopefully informative posts, questions and answers. If people come here looking to learn something about Blender or film-making, I don’t think it’s good that they have to read through all of this.
Anyway… Thanks as always for the nice comments!
Fweeb: Ah, well it looks like you answered your own question :) The option in the OOPS is just to let you visualise what is linked and what is local, to help keep track of it all. The outliner now also draws linked data with the [Li] icons, too.
Kattkieru: Well, We’re using a separate .blend file for each camera change, yes. We’re using grouped objects for sets and props, linked in from central library files. In many files in the same scene, the lighting might be pretty similar, so we can just append lighting setups from the other shots in that scene, and tweak them locally, but we need to have the flexibility to do this i.e. not linking the lights.
Fowlalgorn_gui: Thanks! We talked about the translations over dinner last night. We definitely want lots of languages on the disc, maybe when it’s all a bit closer to completion we will post a request for translation help. How we will manage it (giving people the script, checking for correctness, etc) is a bit difficult, but hopefully we will sort it out soon when the time comes.
efbie: Yes, we’ve had some experiments with Verse in the studio here. It was interesting, but we haven’t been able to use it for production work yet. Toni recently wrote a report about it for the uni-verse group which they are taking into consideration, and Jiri Hnidek is working on verse image support in Blender right now (for Blender GIMP image connections) so maybe we might still use it yet.
Oh one more comment about the seperate .blend files for each camera change:
In a scene where the characters are talking/acting across several shots, one could take two approaches: one blend file with continuous action , or several for each shot.
The first style I’ll call the “documentary” approach; you let the characters do their thing, and film them. The second style is more trickery; the reason is that you can tailor the action for the shot.
This means, with good animation, that the actions aren’t really continuous across shots, they only look that way- they are all staged for maximum impact *from the perspective of the camera*. By having a live edit, the animation director can really make sure that the shots fit, and appear seamless across the edit.
It seems tougher, but it saves heaps of time in practice. We’ve even managed (finally) to get our animation styles to fit, so it’s not readily appearant to the viewer that the shots are animated by different animators.
I think you are very unwise in thoughts and words Mr Jongle.
I let pass by.
Matt: You´re right. Sorry about that. Btw. I just love these progress posts. Wonderful that you all take the time to show us detailed steps of your progress and workflow.
Best wishes
/JoOngle
I apologise to community for mentioning thoughts of monies for Blender in frustrated conversation before about posting standards on Orange.Was to be anonymous anyway so nothing to do with ‘suck up’, leverage or get credit for. I forget myself in moment and make blunder saying about.
I state for you now I decide officially to reassign this sum I thinking of to already receiving aid organisation for very needy persons worldwide so can be no false expectation arising.
yours rubbleman
Awesome!!! I have been wondering what you wer doing for versioning and collaboration. Sounds very efficient and effective.
Matt: I have a question on what you just said about appending the lights. So, when you link the group can you make local changes to the grouped objects or are they static like library linking? It almost sounds like groups are like appending. What is the major difference?
Wait. I just reread the explanation again is this it: Linking groups is just like linking objects etc. except for you can add/change objects in lib files that are in the group and instead of linking the new objects just created, all files using the “group” will have the new objects added?
Genbod: Precisely! Groups can be either linked or local, just like Blender objects. We happen to be using a lot of linked groups because it gives us a lot of flexibility, and makes things much more convenient to manage since a prop or set piece may contain many objects, of different types.
We still haven’t finalised lighting yet, but most likely they’ll all just be individual local lamp objects per file that could be appended locally and tweaked to save a bit of setup time.
Hey guys,
I think its super sweet that you all are doign a big Blender project like this, and like many others i cant wait to see what you guys will be cranking out.
Very best of luck to you, you have some very talented ppl on the team, so it should rock quite sufficiantly!!! :o)
Matt
Everything you guys do is cool.
I see a lot of great info in this blog.
Have you considered adding it to the DVD content?
john
I cant find the group function in the “new” Blender 2.41….. where iis it`?
Silverframe,
iirc it is only in current cvs (ex orange branch), and hence coming to 2.42 – like all ‘orange features’ (like compositing and seq mem cache limitor (developed outside orange but used by us))
~Toni