www.BinaryAlchemy.de :: View topic - multi proc rrCreateVideo
 SearchSearch   RegisterRegister  ProfileProfile   UsergroupsUsergroups   Log inLog in 
If you create a new post, please use a topic that describes your problem
Documento sin título
 
multi proc rrCreateVideo

 
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    www.BinaryAlchemy.de Forum Index -> old - RR Plugins/Extensions/SDK/configs - v6.x
View previous topic :: View next topic  
Author Message

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Mon Apr 12, 2010 5:25 pm    Post subject: multi proc rrCreateVideo Reply with quote

We have artists rendering 25,000+ frame seq and would like to have a movie at the end for review.

Yes one machine can make a movie of 25,000 frames but it takes a long time.

I am interested in creating a PostScript mkmovie (a.k.a. makeMovie) that does the following steps:

- assign each machine a 1,000 frame section to create a movie (25,000 frames = 25 movies of ~ 1,000 frames each)
- merge each of the 1,000 frame movies together to make one final movie.

This would create a movie 25 times faster than the current setup.

What I don't know how to do is to create a postscript that would spawn new jobs for multiple machines with one process watching them all. If a process gets interrupted launch on a new machine. And when complete knows to merge them together.

Have any thoughts to get me started, seems very possible, I might need some wise words from you and how to best interface with RR 6 to get this going.

Thanks,

mdreams
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Mon Apr 12, 2010 5:41 pm    Post subject: Reply with quote

Hi
I am sorry, but post-processes cannot be split onto multiple machines.
It is a one-machine task.

PS: I am not sure if you can append .movs together.
I only know it from .mpeg.
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Mon Apr 12, 2010 7:19 pm    Post subject: Reply with quote

I will poke around some and get back to you....

thanks

FYI it is possible to append .mov files together.
Back to top
View user's profile Send private message Visit poster's website

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Fri May 21, 2010 10:57 pm    Post subject: Reply with quote

How does one launch a "Pre-Script"?

I see "post" and "done-scripts" here: <RR>\render_apps\_prepost_scripts

Not sure where an example is or what a "Pre-Script" should look like.

Also...

What are your thoughts on how we could create a multi-proc (multi-processor/machine) makeMovie?

Currently we are working on creating a new "Renderer" for RR and use a pre-script that reads in a job and creates an additional render job in RR dependent on the first jobs that will create a movie using multiple machines.

(I hope this makes sense hard to explain in written word)

Any comments on our strategy?

thanks,

mdreams
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Sat May 22, 2010 11:08 am    Post subject: Reply with quote

Hi

They look the same, you just have to set
PrePostType=pre

Not sure that I understand your workflow right.
Why do you want to create a pre-script to read job information?

Perhaps you can describe more what you want to execute in which order instead of directly translating it into RR procedures.
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Mon May 24, 2010 4:09 pm    Post subject: Reply with quote

Thanks for the "PrePostType=pre"

Let me explain in long form.

Problem: it takes too long for makeMovie on one machine to create a movie from a 2500 frame sequence.

Solution: Use multiple machines that each create a movie for a different frame range of the 2500 sequence.

For example: use 10 machines that each make movies of 250 frames each and finally merge all 10 movies back together for a movie of 2500 frames.

This would take ~1/10th the amount of time compared to one machine to accomplish the makeMovie task.

How do we accomplish this inside of RR?

Current plan:
Create a pre-script that will create a new job in RR that will execute the multi-procMakeMovie command.

How this might look in practice:
User submits a comp job of 2500 frames, and checks our pre-script "multi-procMakeMovie" command.

The pre-script reads the "2500 frame comp" job and creates a new job, dependent on the original, called "multi-makeMovie 2500 frame comp" and drops this new job into the "autoload" dir. The "multi-makeMovie 2500 frame comp" waits for the "2500 frame comp" job to complete and then executes giving the user a movie much faster.

Problem with current plan:
The pre-script will not create the dependent multi-mkmovie job until the first job has started. which means many jobs could be submitted between the submission of the initial "2500 frame comp" and when the "multi-makeMovie 2500 frame comp" job is created. The only way to combat that would be by raising the priority/rank of the "multi-makeMovie 2500 frame comp"


Thanks for any feedback or thoughts you have.

mdreams
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Mon May 24, 2010 4:50 pm    Post subject: Reply with quote

Hmm, the "wait for" is an issue which can not be solved at the moment.

I think it would be better to use a done-script.
When the comp job has finished, the last thing it executes will the the submission of the quicktime job.

So it is practially the same as wait for as the done-script is executed directly before the job is finished.
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Mon May 24, 2010 7:54 pm    Post subject: Reply with quote

Thanks for that. We will look at using the "Done-script"
Back to top
View user's profile Send private message Visit poster's website

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Tue Jun 01, 2010 10:02 pm    Post subject: Reply with quote

We have created a multi proc MkMovie as a new renderer inside of RR using a new config file in \render_apps\_config. It can be submitted just like any other job inside of RR. Inside the config file, it uses <SeqStart> and <SeqEnd> to specify the frame range and then executes some external python code that makes the movie slice. This works and is multi proc'd over our network.

The problem that we are now having is that once a machine finishes a frame range and executes <CheckExitCode>, the same frame range is then place back in the queue and is processed on another or even the same machine. The same thing happens if we use <CheckExitCode> <FN> after completion. The render is marked complete and resubmitted.

I noticed that after completion and execution of <CheckExitCode> <FN> that in the job log RR says "
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++++++++++++++++++ Render Executable done +++++++++++++++++++++++++++++++++++
++++++++++++++++++++ Royal Render checks the return code of the executable ++++++++++++++++++++
++++++++++++++++++++++++++++++++++++++ 06.01 14:03.59 +++++++++++++++++++++++++++++++++++++
++++ Executable returned 0 (0x 00000000) as exit code for frame 1.
++++ Render successful
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
"++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++"

This was on a frame range of 1-67. It says this and then queues the same range back up to RR. How do I specify that the frame range (1-67) is complete and does not need to be rendered again?

mdreams
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Tue Jun 01, 2010 11:45 pm    Post subject: Reply with quote

Hi

The CheckExitCode is running fine. It tells the client that it was successful.
The issue that the same frame range is send to a different machine is probably that the server cannot find the output of this "render".

If you can no provide an output file for the render, then set
DoNotCheckForFrames= 0~1
in the render config.
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Wed Jun 02, 2010 4:55 pm    Post subject: Reply with quote

Hello again,

I have added DoNotCheckForFrames= 0~1 to the render config, but the problem still persists. Is there a way to specify what the output file for the render is, or is there any other way to prevent resubmission of the same job other than DoNotCheckForFrames= 0~1?

I also have noticed that in addtion to repeating, that the sequence is being divided in a weird way. For instance I ran a test just now on frames 1-30, and frame ranges that RR sent to clients were 1-10, 11-20, 21-26, 21-30, and 27-30. These ranges are coming from <SeqStart> and <SeqEnd>, but I do not understand why frames are being reused since RR has already added them to another computer's slice. Any suggestions?

Thanks for your fast responses, your support has helped us to come very close to completion of this task.

mdreams
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Wed Jun 02, 2010 6:02 pm    Post subject: Reply with quote

Hi

There are only these two possibilities.
Either define the right output at submission or enable "DoNotCheckForFrames".

Please recheck the job in rrControl if DoNotCheckForFrames is enabled.


"why frames are being reused"
The server should not resend the same frame as long as one client is rendering on it.

Please select the job in rrControl, create debug information via the menu "debug" and upload them via www.RoyalRender.de/upload.php
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Wed Jun 02, 2010 6:47 pm    Post subject: Reply with quote

Upon checking I realized that DoNotCheckForFrames was in fact disabled. I had to close and reopen the submitter and then the changes took effect. All the pieces rendered this time with no repeated frames. However now there is another problem. When a client crashes or manually aborts their job, the frame range that the client was assigned is not being resubmitted by RR. Is this a side effect of enabling DoNotCheckForFrames?

I will upload the debug info as soon as I am cleared to release the it.
Back to top
View user's profile Send private message Visit poster's website

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Wed Jun 02, 2010 7:20 pm    Post subject: Reply with quote

I have uploaded the debug file.
Back to top
View user's profile Send private message Visit poster's website

schoenberger
Site Admin


Joined: 02 Mar 2005
Posts: 3786

PostPosted: Wed Jun 02, 2010 9:39 pm    Post subject: Reply with quote

Ok, the issue is the preview rendering.
At first every 33 frame is rendered.
Which makes no sense for your render script.

Please add
RenderPreviewFirst= 0~0
to the render config.
_________________
Holger Schönberger
Binary Alchemy - digital materialization
Back to top
View user's profile Send private message Send e-mail

mdreams



Joined: 28 Sep 2006
Posts: 215
Location/Company/Country: US

PostPosted: Thu Jun 03, 2010 6:19 pm    Post subject: Reply with quote

We have set "RenderPreviewFirst= 0~0" but still RR keeps remaking each movie slice.

Of course DoNotCheckFrames does keep the movies from being remade but also would mean if a user submits a job with the "Abort Jobs with lower piority" it would abort our mkMovie job and would not restart. We need a more robust solution than DoNotCheckFrames.

RR is creating the movies we tell it.

For example:

testMovie.1-50.mov
testMovie.51-100.mov
testMovie.101-150.mov

How do we tell RR this is the name of the movies (listed above) we are making so it does not try to remake? How do we tell what RR is looking for? (obviously we are failing at telling RR what to look for, else it would not remake)

thanks for your help on this.

mdreams
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
This forum is locked: you cannot post, reply to, or edit topics.   This topic is locked: you cannot edit posts or make replies.    www.BinaryAlchemy.de Forum Index -> old - RR Plugins/Extensions/SDK/configs - v6.x All times are GMT + 1 Hour
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
 
Documento sin título
 



Powered by phpBB © 2001, 2002 phpBB Group



Number of shameful bots caught by Anti-Spam ACP: 1667