PyFarm — Distributed Python Rendering

By my own error the installation instructions said to use gksu when it should have said to use sudo. While this error was small and did not warrant its own release I also had to remove several test functions from the program that prevented it from launching. The updated code has been posted to the releases section as well as on HighEnd3D.

On a side note, if you have sent me a report via the contact form I got your message (thank you) but I did not get your email address. Due to an active security update of the subsystem that handles outgoing mail the sender’s addresses were removed from each message. This issue has been corrected and all future messages should pass without issues. If you still wish to receive a reply from me, feel free to drop me a line.

Page Added:

• Linux Installation — Red Hat instructions to follow

Page Updated:

• Requirements — Changed Python version requirement

Stay tuned!

Version 0.3.214 is now available for download in the releases section.

Click on the image below to watch the video of v0.3.214 in action.

As my final quarter drew to a close at SCAD I was able finish v0.3 however due to time constraints I was unable to build a package in time to post to the web. This shall be a short term problem however and the update will be made available soon.

Along with this update there are two other development announcements for PyFarm. First and most importantly I am announcing the commencement of development towards PyFarmCE or Corporate Edition. This addition will be an add-on available to companies, workgroups, and advanced users that wish to use PyFarm in a centralized environment. More information about its features and specifications to follow. Lastly I would like to state that future updates to PyFarm and PyFarmCE should be viewed from the rss feeds, roadmaps, and wiki pages. This blog will be updated only during major releases or upon completion of a new research.

Thanks and happy rendering!

Revision notes are a useful way of tracking changes between revisions. In the past in order to view these notes you have either had to visit PyFarm’s launchpad website and dig down the find the revision notes or checkout the working branch itself. The latest revisions page provides a great deal information about development including new bugs, bug fixes, feature implementation, and much more. For your convenience these reports also include a link the original launchpad tree, date committed, and revision number. Click the link below to view the past ten revisions.

About -> Latest Revisions

Socket based communication falls into one of two primary categories, TCP or UDP. TCP or Transmission Control Protocol is used for many things including loading html pages, transferring files over the internet, and downloading email. UDP or User Datagram Protocol is used when the constant flow of data is more important than each individual packet. Examples include instant messaging, instrument monitoring, and online gaming. Overall UDP sockets are easier to use and implement but less reliable over time (see Table A for complete details). However when it comes to performance UDP is essential because it offers a connectionless (aka stateless) connection. This simply means that although data is being sent to the server the server does not reply back to the client with the packet state. This stateless connection leads to much lower latency and lower processing times making each individual connection faster. For something like process logging where the processes cannot be stopped or paused UDP offers the perfect solution because it does not wait for the connection to start, transmit, and properly terminate. In the upcoming release of v0.3 the job logging is performed by UDP socket connections, you can read more about TCP and UDP sockets on wikipedia

Table A.
TCP vs. UDP Packets

TCP	UDP
End to end connection (connection oriented) Error checking Stream oriented Guaranteed Delivery Bidirectional Moderate performance	No end to end connection (connectionless) No error checking Packet oriented Delivery not guaranteed Bidirectional High Performance

The third and final release candidate is now complete and available for download.  As this is a release candidate please be sure you read the associated wiki page before proceeding with your download.  Also included with this release is small readme file with some very installation instructions.

[caption id="attachment_521" align="alignright" width="239" caption="Figure B"][/caption]One of the major problems when working with large sets of data is trying to display the information in a timely manner. As an example a user requests a 300 frame render on a maya file that contains four separate render passes resulting in 1,200 frames to process. In addition to those 1,200 frames there are at least eleven individual attributes that keep track of the frame’s information and status bringing the total number of data points up to 13,200. To generate, query, assign, and filter this data into an interface before displaying it to the user takes at least ten seconds each time the information is requested. This is far to long to wait each time for the interface to refresh so a better solution to this flow of data (as seen in Figure A) had to be found. After some extra research I came across the idea of Model/View programming, an architecture used to define the relationship and interaction of data with the interface. In the previous design the interface had to wait on all of the information to be assigned before rendering the interface itself, however by breaking this down into smaller steps massive sets of data can be displayed instantly. For example upon opening the job details interface only about eighteen frames will need to be displayed so instead of requesting 13,200 points of data only 198 need to be determined then mapped to the interface. This is where the new model comes into play and has many advantages over the previous design. When the user requests more frames by scrolling the data is then requested by the delegate which asks the model (which itself has the prepopulated data) which in turn informs the main table of the new frames. While this development has delayed the release till at least Mondy further delays are not predicted at this time.

Several things have changed in the past week, most of which marks a major shift towards the final product. To start off with the 1300+ lines of code from the main program have been shifting into external libraries resulting in much cleaner and easier to read code. Also I have been working on developing methods to deploy PyFarm to systems other than Linux (code was already cross-platform, deployment has been the issue). As a result of this research an installer now exists for Mac OS X which will be deployed along side the third and final release candidate. Lastly, almost all status related events have been moved into an event based framework further reducing the amount of code required to inform the master of a client’s status. Stay tuned, release candidate three due out in about a week!

After a couple of days working to convert the old data management system info the new dictionary based system I have been able to provide the user with host information. This will allow the user to see specific info and stats about the selected node including IP, hostname, status, os, architecture, frames rendered, frames failed, frame failure rate, and software installed. Future versions of this widget may allow the user to disable rendering for specific software packages on a per node basis.