Hi,
This is a discussion that started in the bugs forum, and I am moving it here, as Bobo suggested.
Thanks 
Hristo: Don’t you guys think that file operations should be encapsulated and abstracted to the user? I imagine the user interacting with a tree structure - level 1 for animations, level 2 for frames, level 3 for partitions. So it would be easy to manipulate everything. The way it is now is nice for an in-house tool, but a shelf product should have an extra interface layer.
Bobo: I will ask Mark Wiebe to discuss this.
It is the way Digital Fusion deals with files and the way we wish MAX would deal with sequences but it does not. There are significant advantages to how it is now.
I would suggest posting a separate thread about this in the General Discussion forum…
Ok, we can see the need for encapsulating separate Partitions in sub-folders.
We can also imagine different users preferring either option, so I intend to add the OPTION to write partitions to sub-folders of the destination specified, then we can compare and either leave both (or more than two) options to store data, or pick the one that is best from everybody’s experience.
Since this is mostly a GUI thing, I think I can implement it very soon…
Could you please write down in great detail how YOU would like it to work? (Sort of like a specification).
Thanks,
Bobo
Bump!
Still waiting for ideas (from everybody) how to simplify, optimize, extend or whatever the file management in Krakatoa.
Current system (as of 0.9.7)
*You can use the […] button to enter a save name and path.
*Extension and trailing underscore will be added automatically.
*[E] button allows you to explore the location.
*You can type in or paste ANY path - if the path does not exist, the user is prompted about creating the path.
*If the path cannot be created, the path will be reverted to the last valid one.
*Again, extension and trailing underscore added automatically.
Ideas for SIMPLE improvements:
*Separate ROOT from SUB-DIRS by using multiple text fields.
**The ROOT would be the disk location where most Krakatoa file sequences are written to. (usually, a large disk either on your machine or, as it is at Frantic, a huge disk array for just particle work).
**The SUB-DIRs could be one or more - for example, a PROJECT name and optionally a particle set name. This way, you can enter a root path once, a project path whenever you start a new project, and then just change a field with the actual name of the particle sequence.
*For each text field, a history feature could be added to remember previously used paths, so you can go back and forth between anything that has been used (stored in INI file).
*A button for incrementing TAKES - at least at Frantic, each iteration gets a numbered take number in the form “tk123”. The actual format pattern could be customizable, and a button could be added to increment with a single click just like the [+] button in the Max save dialog.
So the save file sequence area could look like
[H][…][c:\temp\particles][E]
[H][projectname ]
[H][sequencename ][tk123][+][+][+]
[H][baseFileName_.prt ][M]
where
[H] is a history button to open a browser with all previously used values;
[…] opens the save file dialog (done);
[E] opens the Windows Explorer (done)
[+] increments the take (with major, minor and iteration increment -> the first [+] would make tk123 into tk200, the second into tk130, the third into tk124). The “tk” prefix would be user-defined.
[M] gets the Max file name (if any) and places it into the base name field.
This does not really abstract the file organization, but provides UI level tools to work easier with a large number of particle files without having to hand-edit a long full path each time. Parts of the path can be rather persistent, other parts can be easily changed to create version increments or new sequences.
At the end, Krakatoa would combine all these fields into a single path to write the current output to (like it does now).
At this point, I would prefer keeping all partition files in a single folder, but we still have the option to create a directory structure below the \sequencename directory with sub-folders for each partition.
Borislav “Bobo” Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
Awesome, just the way I wanted it, and more!
Would it be possible for Deadline to use those fields in the job name or description?
>Would it be possible for Deadline to use those fields in the
>job name or description?
Certainly! :o)
Cheers,
Borislav "Bobo" Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
Oh crap, I need to stop asking such open ended questions.
Why?
It was a great idea and took 3 minutes to implement.
In the Deadline rollout, now there are two buttons [<-Path][<-Scene]
If you press Path and you have set the Save Path with Project and Sequence (and optional Take or Version), the Job name will be set to
Project_Sequence_takeNumber
If you press Scene, you will get the Max file name as before. Then you can edit them to add more stuff.
I can add the Path button to the Comment field, too, if you prefer to use Max file name in the Job Name and the Project/Sequence/Take in the Comment.
Thanks for the ideas!
Cheers,
Borislav "Bobo" Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
I meant that my vague question led to your answer which didn’t indicate if the idea would be implemented, only confirming that it was indeed possible.
Don't worry, we try to read between the lines.
In the mean time, I expanded this feature to use a context menu - there is now a [...] next to the Job Name and the Comment field.
When you click it, a context menu pops up and gives you the choice to select the Max File Name or Save Path Project_Seq_Take form, and also to set the default behavior for the future, so you can customize it to always use the Save Path for the name, or for the Comment, or even in both...
This way, we can expand this in the future with more options without cluttering the GUI with more buttons with ugly <- symbols.
Thanks again for the idea!
Cheers,
Borislav "Bobo" Petrov
Technical Director 3D VFX
Frantic Films Winnipeg
I believe some of these points have been addressed since I last gave Krakatoa a push through production several months ago, but I’ll post here for sake of discussion.
Thanks,
.jeremy
***
When it comes to rendering particles that are close to camera we’ll often find ourselves in a situation where we must render very heavy batchs of particles to resolve to somewhat smooth results. The current workflow currently requires meticulous care when setting up the particle cache and subsequent xml jobs. I’d like to see this rather involved two-stage process consolidated into simple single stage setup - where 100% of the settings can be defined in a single submission - a submission that can ideally be transfered to a remote office with minimal difficulty and time.
Here are a few rough concepts I have in mind to achieve this:
A.) Output/Input paths/Deadline prefix name etc could be consolidated in a central area on the Krakatoa interface, too many windows need to be accessed while setting up jobs.
For instance, when setting up XML jobs, roughly 4 main fields need updating prior to submitting. A.) Particle Files to load B.) XML files to save, C.) Exr images to output D.) Deadline job name. On average we’d have 10+ jobs to submit like this, over VNC - very time consuming. Home directory where default object-cache, particle cache paths and XML files are automatically defined/set. The intent would be to streamline the output/input process for the user as preventing user error - Advanced tab would still provide ability to potentially modify any values if necessary.
B.) Automatic particle-cache/Partitions slave load balancing. I envision a specialized managed pool of slaves where a single Krakatoa particle scene is evenly distributed allowing a greater quantity of machines to cache and render at once. Currently, unless we manually split them up onto multiple servers (possible, but one more thing on top of already complicated process) we’re really only safe to have 10 render-nodes writing/saving to the particle-slave server at any given time. No doubt this could be quite a challenge - but it would save us from a lot of headaches.
C.) A continuously refining Krakatoa render. Basically I’d imagine a system where we submit one Krakatoa scene where it sits rendering on Deadline indefinitely while continuously spitting out more and more passes, automatically combining the results until a smooth result is achieved. During this process as the cache has been fully utilized in a successful krakatoa render/pass - it’s automatically deleted from particle slave’s storage as it’s no longer needed, freeing up space for the subsequent auto generated iterations.
***
Thanks, Jeremy.
Please note that the Max version of Krakatoa does not have XML support (yet), so most of your points do not apply to the commercial product we are trying to prepare for shipping.
Cheers,
Borislav "Bobo" Petrov
Technical Director 3D VFX
Frantic Films Winnipeg