What’s New In Deadline 8.0.0.? (Beta 1)
Deadline 8.0 is the latest version of Thinkbox Software’s scalable high-volume compute management solution.
For ultimate scalability, Deadline 8 introduces on-demand metered licensing, which will work with both local and cloud-based farms alongside existing permanent and temporary licenses. Usage is tracked per minute, and can be purchased and managed round-the-clock via Thinkbox’s e-commerce portal. Additionally, the store will host on-demand per-minute licenses for select third party content creation applications and plugins
Deadline 8 also adds a Proxy Server application that allows users to securely connect to and interact with remote office and cloud-based render farms over public Internet without the need for a virtual private network (VPN). An updated user interface with enhanced interactivity improves the Monitor experience, while the new sandboxed Python environment facilitates additional rendering and event stability.
Here are the highlighted changes:
On-Demand Metered Licensing
For ultimate scalability, Deadline 8 introduces on-demand metered licensing. The feature will work with both local and cloud-based farms alongside existing permanent and temporary licenses. Usage is tracked per minute while the Slave application is running, and can be purchased and managed round-the-clock via Thinkbox’s e-commerce portal (coming soon). Additionally, the store will eventually host on-demand per-minute licenses for select third party content creation applications and plugins.
Deadline Proxy Server
The new Deadline Proxy Server application allows users to connect to and interact with remote office and cloud-based render farms without the need for a direct connection to the Database or Repository. The Proxy Server implements a standard REST communication protocol, allowing existing systems like Apache and Nginx to be used for SSL security and load balancing.
For example, by using Nginx and SSL security, users can connect to a cloud-based farm over public Internet without the need for a virtual private network (VPN). In addition, multi-location studios can also use the Proxy Server within their VPN to connect to Repositories in other offices without having to mount the Repository server locally. This is especially useful over high-latency network connections.
Improved Monitor Interactivity
The Monitor now remains fully interactive while loading and updating tens of thousands of jobs and thousands of slaves and data loading and updating is much quicker as well. This was accomplished by moving a lot of the list processing code to native C++ libraries.
The Monitor also remains fully interactive when processing long operations, like deleting thousands of jobs. Long running operations are queued up and shown in the new Background Operations panel, and can be cancelled or paused to allow other background operations to proceed.
Easily Launch Multiple Monitors
The Launcher and Monitor now have options to launch additional Monitors connected to other Repositories. When launching a Monitor that is connected to another Repository, it won’t change your default Repository unless you choose to, which means that other Deadline applications like the Slave won’t accidentally connect to the wrong Repository when they launch.
Asynchronous Balancer Operations
The Balancer now performs the main VM balancing operation in a separate thread. This thread can be interrupted if the interval between balancing operations has passed, or if the Balancer is being asked to shut down. This update allows the Balancer cycles to be performed at more regular intervals than before.
Python Sandboxing
Render plugins and events now run in separate processes from the main Deadline applications. This improves stability because crashes or deadlocks that occur in the render plugins and events no longer affect the Deadline applications that are executing them. For example, if the render plugin that the Slave has loaded crashes, only the sandboxed Python process will be affected, and the Slave will detect this and recover by simply starting up a new sandbox process.
When rendering, the sandboxed process’ initial environment will match the environment variables specified for the job (if there are any). In addition, the environment for render plugins is now reset between jobs, and modifications to the render environment no longer affect the Slave application’s environment. This means that environment variables set by the render plugin for one job won’t affect the next job that the Slave picks up.
Local Python API Synchronization
3rd-party Python APIs that Deadline uses are now installed and synchronized locally to the Client machines. This improves the performance of any plugins (rendering, events, cloud, etc) that rely on 3rd-party libraries because they can now be loaded off the local machine instead of from the Repository. The upgrading of Python APIs is handled by the same system that Deadline uses to perform automatic upgrades of the Client software.
Plugin and Event Configurations in Database
Rendering plugin and event plugin configurations are now stored in the Database instead of the dlinit file (similar to how Cloud plugin configurations are stored). In addition, Rendering plugin icons are now stored in the Database, and can be set via the Plugin Configuration dialog in the Monitor.
Opt-in Events
Event plugins now support a third “Opt-in” state, in addition to the existing Enabled and Disabled states. When an event is in the Opt-in state, slaves and jobs will only trigger the callbacks if they have “opted in” to the event plugin. Opt-in events can be specified for jobs in the job info file, and these events can also be configured from respective property dialogs for jobs and slaves in the Monitor.
Event plugins also support a new EventCallbacks entry in the plugin configuration that specifies which callback(s) the event triggers for.
Updated to MongoDB 3.0.6
Deadline now ships with MongoDB 3.0.6, with version 3.0.0 being the new minimum requirement for Deadline 8. Deadline utilizes MongoDB’s new collection-level locking functionality, which simplifies the data layout because data in the database no longer needs to be split up into separate databases.
General Improvements
- Added a new Proxy Server application that the Deadline applications can connect to instead of connecting directly to the database and repository.
- 3rd-party Python APIs that ship with Deadline are now installed and synched locally to the Client machine. Any plugins that uses these APIs now load them from the local location instead of from the Repository.
- Pulse settings, Balancer settings, and Cloud Regions can now be imported from another Repository in the Monitor.
- Regions are now defined at the Deadline Client level, instead of being a property for users, slaves, etc.
- Added a new licensing option to use metered licensing instead of permanent and temporary licenses.
- All submission scripts and plugins have been updated so that they work when the Deadline applications are connected to the Proxy Server instead of the Repository.
- All Monitor scripts now use icons that are stored in the Database, instead of loading them from the Repository.
- Slaves can now have arbitrary key/value pairs associated with them.
- Added a configurable soft cap to Limit Groups, an event that get’s triggered when it is exceeded, and an event that triggers when a Slave that exceeded the cap returns the limit.
Installer Improvements
- Added a check to the Repository installer to detect if the local loopback IP address (127.0.0.1) is being specified for the database server.
- Shortened the name of the daemon init scripts that the Deadline installers create on Linux.
- Added a setting to the Repository installer to allow users to set open permissions on the Repository folder.
Database Improvements
- Upgraded the version of MongoDB that Deadline ships with to 3.0.6.
- Data is no longer divided into multiple databases because MongoDB 3.0 now supports collection level locking.
Job Improvements
- Added extra info properties for tasks, and the labels for those properties can be optionally configured in the Repository Options, and overridden in each job’s properties.
- Added frame awareness to the Asset Dependency property for jobs. If enabled, Slaves will now check the existence of assets required by specific frames for a task before dequeuing the task.
- Jobs now keep track of the Region they were submitted from. By default, it’s based on the Region of the Client machine that submitted the job, but it can be overridden in the job info file on submission.
- Added option for auto-job cleanup to skip jobs that have errors.
- Add option to Jobs to be interrruptible if their task progress is below a configurable threshold.
Client Application Improvements
- All Deadline applications now pull their Region from the machine’s Client settings (regions are now defined at the Client level).
Launcher Improvements
- Added an option to the Launcher menu to launch a Monitor connected to a specific Repository.
- Added an option to the Launcher menu to change the Region for the Deadline Client (regions are now defined at the Client level).
Monitor Improvements
General
- Events are now sandboxed from the main Monitor process.
- Events are kept loaded in memory to improve performance, and events are processed in a background thread to keep the Monitor interactive when events are triggered.
- Improved list processing performance by moving some code to native C++ libraries.
- Long running operations are now processed in a background thread, and can be viewed, paused, and cancelled from the new Background Operations panel.
- Right-click script menu items can now have multi-selection enabled or disabled, and some script menu items can be enabled or disabled for certain plugins, states, etc.
- The option to import settings from other Repositories now supports Pulse settings, Balancer settings, and Cloud Regions.
- Advanced filters for data panels can now be set in a new side-panel instead of in a separate dialog. The filter panel’s visibility can be toggled on and off for each data panel.
- Added Task Extra Properties tab to the Job Settings page in the Repository Options, which can be used to give meaningful names to a job’s extra task info settings.
- Added an option to the File menu to launch another Monitor connected to a specific Repository.
- Icons for plugins can now be specified in the Plugin Configuration dialog.
- Added option to Cleanup tab for Job Settings in the Repository Options to not cleanup jobs that have errors.
- Added -proxyserver command line option to the Monitor to have it connect to the specified Proxy Server at startup.
- Fixed a bug when saving and loading states that contain unicode characters in user, filter, or layout names.
- Added option to Tools menu to download installers for the integrated submission scripts.
- Fixed a bug that caused a Deadline Monitor.ini file in the user settings folder to slowly grow indefinitely over time.
- The Monitor Scripts customization interface has been moved from the Repository Options into its own dialog.
Jobs
- Job Batch groupings now show more aggregated data in the Batch row.
- Job Batching can be enabled or disabled for each Job panel separately.
- Added Monitor Option to enable or disable the grouping of Job Batches that only contain a single job.
- Added buttons to Job panel to expand or collapse all Job Batches.
- Quick filters for jobs now only contain options for pools, secondary pools, groups, users and plugins that actually have jobs associated with them.
- Added Task Extra Info Names page to the Job Properties dialog, which can be used to override the names in the Repository Options.
- Added Event Opt-in page to the Job Properties dialog, which can be used to allow jobs to opt-in for non-global events.
- Resuming failed jobs is now enabled in the context menu if a job has any failed tasks.
- Added a column to show the Region that the job was submitted from.
- Scanning for missing output now works when multiple jobs are selected.
- Menu items to change the state of jobs are now all available when selecting multiple jobs that are in different states. Only jobs that are in applicable states are affected.
- The estimated time remaining for jobs is now shown in a new column in the job list.
- Add ‘Find’ functionality to the Submission Params’s text boxes in the job properties dialog.
Tasks
- Added columns for Task Extra Info.
- Added Quick Filters to the task panel.
Job Dependency View
- The graph now shows nodes for Opt-in Events that the job will trigger, and Opt-in Events can be added and hooked up as well.
- The graph now refreshes faster and is more interactive when there are lots of nodes.
- The graph is no longer redrawn when clicking on jobs that are already in the dependency tree, and the layout of the graph is now the same regardless of what job is clicked on.
- The graph now groups job, asset, and script dependencies together.
- Added an option to use straight or curved connectors between nodes.
- Improved the layout of common dependency trees, like linear trees, one-to-many trees, many-to-one trees, etc.
- Increased the default canvas size of the graph to make panning easier.
- Added a configurable maximum for the amount of items (nodes and connectors) that can be displayed.
Job Details
- The Job Details panel now remembers which groups are expanded and collapsed when clicking on different jobs.
Slaves
- The Slave panel now shows when a Slave is performing house cleaning, and Slaves in this state have a unique color.
- Added Quick Filters to the slave panel.
- Slaves can now use metered licensing, in addition to permanent and temporary licenses.
Cloud
- The Cloud panel performs much better when connected to a remote Repository because the Cloud plugins no longer load their 3rd-party Python APIs from the Repository.
Slave Improvements
- Render plugins and events are now sandboxed from the main Slave process, so if they crash, the Slave application is unaffected.
- The environment for render plugins is now reset between jobs, and modifications to the render environment no longer affect the Slave application’s environment.
- Added checks for keeping a sequential job loaded if a Slave encounters an error when trying to dequeue the next task.
Pulse Improvements
- The built-in pulse web service has now been removed. Please use the standalone web service application that was introduced in Deadline 7.
Balancer Improvements
- The main balancing operation is now performed in a separate thread that can be interrupted if necessary.
- Fixed some issues with the Balancer icon on OSX.
Scripting Improvements
- Added option for Event plugins to only trigger for certain jobs that “opt-in”.
- Added RepositoryUtils functions for loading and saving the Repository Options.
- Rendering plugin and event plugin configurations are now stored in the Database instead of the dlinit file (similar to how Cloud plugin configurations are stored).
- Plugin icons are now stored in the Database, and can be set via the Plugin Configuration dialog in the Monitor.
- The submission of a job can now be cancelled from the OnJobSubmitted callback by calling CancelJobSubmission() before exiting the function.
3ds Max Improvements
- Added checkbox option: “Use 3dsCmd Plugin” to SMTD -> “Render” Tab, allowing VRay/MR off-load jobs to be submitted optionally to the 3dsCmd plugin instead of 3dsmax plugin. Default set to True. Majority of 3dsCmd specific flags are handled in the expanded SMTD function.
Draft Improvements
- Renamed the Draft plugin and draft event plugin to DraftPlugin and DraftEventPlugin, respectively. This helps avoid conflicts with the “Draft” Python module.
- Fixed case sensitivity issue when comparing codec “dnxhd” in Draft Create Movie Script.
Shotgun Improvements
- Removed System.Drawing import namespace from ShotgunUI.py script, since it’s not needed.
UnrealSwarm
- Added a reserve-type plugin for UnrealSwarm.
Google Cloud Plugin
- Improved some default settings.