Render Farm (Tech Stuff)
While on the subject, I used to have a few PC hooked up through a relatively decent router, but the Render Q would constantly crash when rendering animated frames over the network. It got to the point where the network thing was ditched and the same file was opened on several PCs for individual rendering.
Has this bug been squashed since Poser 7-ish? Is the RQ more more stable for network rendering these days?
If I recall correctly, the early Queue would accumulate in RAM and eventually bog the system down or crash. That was fixed some time ago.
With re-using old hardware in mind I did ponder how it would be to make a Unix compile of the QueueManager application (it does nothing necessarily graphic so should not be difficult) and pack that with a minimal barebone Unix (or DOS64) on a bootable diskette or USB stick. Plug it in. Start system. No need to waste memory on anything else than what you need. SystemHDD can go also.
And oh yes, switch off room heating. LOL!
I wouldn't know if it is feasible to run Queue on Unix or Linux. @shvrdavid might know. If your geek mojo is strong, then you could run the servers on Unix/Linux, and PiXiE boot them all with one command from the workstation.
Note to all: The hardware I suggested is not needed if you only have three or four machines; in that case, you just plug them all into your modem/router (most have four ports)
For the keyboard,mouse, and monitor, you can use a KVM squid ($38) like this:
@Glen85 Do these 10 (?) castoff machines have an OS license? That is, do they have Windows 7 (or Mac OSX) installed and working? If so, is it 32bit or 64bit?
Or, do they have an earlier OS (Win Vista, WinXP, Mac OS 9.6)?
Or, do is the holographic Windows sticker sratched/X'ed out, and there is no operating system installed?
Next question: How much memory is in each machine?
What processor do they have? How many cores? HyperThreaded? Clockspeed of the processor (2.66GHz, e.g.)?
If I recall correctly, the early Queue would accumulate in RAM and eventually bog the system down or crash. That was fixed some time ago.
Good to know. Thank you!
...what about allocating virtual RAM for rendering?
Got a scene that clocks in at ~13GB of RAM. The workstation has enough RAM and the CPU is at a constant 100% capacity:
The slave PC got only 8 GB RAM, and for some reason the CPU is dragging its feet.
Is there a way to remedy this with some hard drive RAM? Or does the Poser scene need to be split up to get it under 8 GB RAM?
The workstation's graph looks good. You want the CPU at 100%.
The slave would do well with more RAM. It is the lack of RAM which is leaving the CPU cores standing around idle. It also puts considerable wear on the hard drive.
Whether you can add RAM is controlled by the motherboard's main chipset; many motherboards are limited to 8GB, some are limited to 24GB, etc.
Server/workstation motherboards tend to have more generous RAM capabilities; even a little midtower ATX server board (like my Urania, an HP z600) can accommodate 48GB. E-ATX boards usually allow 96GB, and 192GB with a RAM riser.
The point is, you need to look at the mobo specs to see how much RAM the mobo can read; more RAM may physically fit in the DIMM slots, but if the main chipset is wimpy, you won't be able to use the extra memory.
Another potential limiter of RAM is the version of Windows. Of course, anything 32bit is severely limited (4GB total, with the video card's onboard memory being subtracted from that 4GB).
Of the 64bit versions, Win7Pro and Win7Ultimate will allow lots of RAM, but the "home" versions not so much. 8GB for Win7 Home Basic, and 16GB for Win7 Home Premium.
You might try smaller bucket size; that will reduce the memory use somewhat (but probably not enough).
You need more RAM. You'll need to look up the mobo's specs to see if it can accept more RAM. Either from the manufacturer's website, or run the free utility Speccy, and in the motherboard tab, look at the chipset vendor & model.
For Urania, the main chipset is an Intel 5520.
Thanks for the pointers... problem is that the slave is a laptop, and 8 gigs of ram is all it'll eat.
Might have to sacrifice the HD for virtual RAM. Possible? Will Poser see it?
@krios I don't know; I've never tried. If the laptop has a second HD slot, and if the second HD can be specified as the virtual RAM drive, I'd get a 250GB SSD ($80) and use that for the virtual RAM, and so spare the OS hard drive.
Anybody else know about using virtual RAM for this?
Ooh, lots to take in, thank you very much!
The machines, at the moment, are totally clean, no OS or anything, most didn't come with hard drives even, so I'm going to use some of the drives I have kept over the years and try moving data around or deleting it, as they all have a lot of stuff on at the moment. RAM wise, I don't know off the top of my head, but as well as these machines, I have an electrostatic bag full of memory sticks to play around with.
I'll try and get as much info on the machines as I can, but they're stashed away at the moment and it involves me turning my tiny flat upside down to have them all out, so it will take some time... maybe weekend. I'm not in a rush to get this thing up and going, just need to do it right. I'm glad it seems like a possibility!
Thank you, all!
@Glen85 Ah, well, the question is, do they have an (intact, not canceled) holographic Windows sticker?
If they have a 64bit Win7 sticker (which version?), you can re-install onto a new hard drive.
If they have no sticker, or the stickers have been X-ed out, then you would need to add the cost of buying an OEM Win7 (or newer) license. These cost about $140 nowadays. The OEM license is cheaper than the retail license, but does not offer Microsoft support getting the OS installed. The install disc runs itself, so I've never had need of MS support.
Before making any decision, you'll want to identify the motherboard. From your description, they aren't operational, so you can't run Speccy. The task is then to open the case side panel, and -with bright lighting and reading glasses- find the model number on the motherboard. We can then look up the specs to tell us that machine's potential.
Regarding memory, if these are a batch of sister machines (same make/model/vintage), then you could cannibalize memory from one to load up another. That option may not be there if the DIMM slots were filled with low-capacity memory sticks. In a case like that, you have to remove the low capacity DIMMs and replace them with higher capacity DIMMs, of the correct type. The RAM sticks must be the type intended for that motherboard, and all of the DIMMs in a machine should have the same clockspeed and timing.
You can look on one of the existing DIMMs, and it will say something like PC3-10600, and there may be a group of numbers like 5,5,5,6. PC3 means DDR3, 10600 is the clockspeed in MHz, and the 5,5,5,6 bit is the timing.
Server/workstation mobos can take either regular memory or registered memory; registered memory does some of its own controller/routing chores, and so a mobo might be specced to take 24GB of regular desktop memory, or 48GB registered memory.
Regular desktop motherboards can only handle the non-registered type memory, and their capacity will be limited, though if it can take 12GB, 16GB or 24GB, then you'll be happy. The remote doesn't need quite as much memory as the workstation; the remote only has to handle rendering one pre-set frame.
Another valuable piece of evaluation data is the type of processor and the type of CPU socket. If these machines have a simple dual-core (two render threads) of modest speed, then that won't offer much render speed. If it has a high clockspeed HyperThreaded hex-core processor (twelve render threads), it will be very helpful as a render slave.
It is often possible to replace a mediocre CPU with a higher performance CPU made for the same socket. Not always, because the motherboard's CPU support features may not be capable of feeding the faster CPU's greater power appetite.
The case may a decal on the grill indicating which series of processor you have (Pentium D, Core two Duo, Core i5, Xeon)
With the machine not running, the only way to find out which processor those castoff machines have is to remove the fin&fan cooler from atop the CPU. Only do this is you're willing to buy a syringe of thermal paste for re-installation. Clean out dust and cat hair while you're at it.
Do ^^not^^ remove the CPU itself.
Use alcohol wipes to clean the grey thermal paste from the CPU.
With a bright light and reading glasses, try to read the processor model number.
Now to re-install the heat sink fin&fan, squeeze out a bit of thermal paste the size of a grain of rice onto the center of the CPU, then replace the heatsink.
If it turns out that the school castoffs don't offer much promise, then you might consider another option, such as buying a refurb server blade.
This Dell C1100 has two HyperThreaded hex-core Xeons -twenty four render threads at 2.66GHz (3.06GHz turbo)- and 72GB of registered RAM, for $351+$35sh.
Install a new hard drive and a Win7Pro OEM license, and you have serious render slave muscle.
Just quickly scanned eBay for "2x X5650"; here's a blade with dual X5650 Xeons (twenty four render threads at 2.66GHz) and 16GB RAM for $150+sh.
That's some really good (if not, in parts, slightly confusing... but don't worry, I'll do some homework) information, thank you. I'll get all the info I can and probably take some pics too, but I'll have to overhaul my flat to get everything out and have room to work, as well as having room to exist in such a place with such clutter, lol! I'll probably number or letter each machine to make it easier to identify them. I'm totally open to gutting everything and having piles of components to make a farm from... more on that idea:
I live in a studio flat, quite small, storage space is hardly existent. My living 'room' and bed'room' are one, but I'd like to separate them. I've designed a modular system which features a built-in bed, divided completely from the rest of the flat... I call it my 'passion pit' or 'pit' for short.
This will be built out of MDF, and it's all entirely doable and wouldn't take all that much work, really. Much of it will be made from interlocking 'boxes', and, as such, could be dismantled in minutes and loaded into a van for moving day, then be rebuilt as a similar kind of thing somewhere else. The 'footboard' of the bed will have a decent-sized television mounted on it, with my studio monitors on each side. The base unit of this is something I've put a lot of thought into... there will be three or four of these 'boxes' stacked on top of each other to make the floor-to-ceiling 'footboard'. The bottom section could, if adequately ventilated, be used to mount the motherboards etc. Cables could be routed one direction or another, keeping it tidy, and ports could be routed to a panel fairly easily, with extensions, if needed.
So, you see, if I vented it all properly and made sure it was all done right, I could have a render farm in one box, with no need for cases, cables etc. If I went for that option, obviously, all the machines would need gutting anyway.
With regards to server blades, I hadn't thought much about that, but that's a decent idea! I have a friend who knows about all that stuff... she spends a reasonable amount of time messing about with servers for work and is Microsoft accredited (or a partner, I don't know, something along those lines), so maybe she could advise too... I'll link her to this thread.
I should just mention that I'm in the UK, though, so getting something shipped could be interesting... not to mention my bank charging me for making payments in non-GBP currency... funny guys!
Thanks for the help here, I'm really pleased... I knew my faith in you lot wasn't misplaced! ^_^
I just fished out some of my hard drives, memory and cards. I do have more but this is what is to hand right now.
Did I mention that I have never, once, 'thrown away' a hard drive... only took one apart when it died? I still have the drive from my very first computer somewhere... it's enormous, lol! And yet, most are full of all kinds of data, from music projects to Poser to po... well, I'll leave it at that, shall I? Lol!
Suffice it to say, I think I'll have plenty of storage space, coupled with the four or five terabytes I'm currently using, for plenty of renders and, if I can figure out how to animate, animations! ^_^
Well, overall, my advice is that before investing money, assess the boxes you have, to calculate whether it is ^worth^ spending cash to get them operational.
The calculus is: How many render threads (at ____ clockspeed), supported by _____ memory, do I get for my investment?
If your ex-school machines have H/T quad processors or better, and have 64bit Win7 licenses, then they are worth buying hard drives for.
I'll note that blades (those flat computers I linked to) are the ne plus ultra of space efficiency, low electrical demand for a given processor horsepower, and can be had with lots of CPU cores and RAM for modest cost.
The one downside is that blades have an array of small diameter high speed fans, so they whistle when running hard in a warm room. Noise is a factor to consider if you'll be doing overnight renders.
Search eBay for UK listings, or search for other UK sources of refurbished servers. Your friend may know of sources.
These are the ones I've been able to get out so far. Some have memory and hard drives, cards etc, others don't. I took the data cables out to give a better view but I'll do more when I'm less drunk and more coherent. Bottom left is my little Compaq Presario, which is a neat little machine. I've not had it running but it has a hard drive and seems complete, so might still have Win 98 on it, lol! It's cute as a button and I love it for what it is, so probably won't include it in the render farm.
Would it be good for me just to take this lot right down to components and work from there, or keep them as they are? I need to test a few of them, as the previous owners reported problems with them which might be terminal, but we'll see. I don't remember which one it was, but the guy said he was certain the MOBO was goosed, which would be a shame, but as I said, we'll have to run tests and see. Neat little project, I think... and I know a motorcycle club which has a 'computer smashing' competition at their annual rally, where they take a sledgehammer to a pile of old cases (nothing of much worth left in them), so I could be well loved in that club again, lol!
On a side note, top right is my favourite case. It looks like crap, but it is easily portable with the handle and plenty big enough for a decent MOBO, so I was considering putting my main PC into that, as this case is an 'Aerocool Strike-X' thing which, frankly, is balls. I hate it, it's the worst case in the world. I'd rather have the gubbins screwed to the wall, lol!
I sleep with Static-X pounding my head to bits through my monitors, noise is fine, lol!
Thanks for all your help, I'll get as much info as I can on it all and get back to you.
@Glen85 Hey! Some of that memory appears to be registered (server memory); I even see what appears to be ECC (error correcting code) chips on a few pieces. Old, though - it's DDR, not DDR2, DDR3, etc. And not big capacity.
Regular desktop memory has four chips toward one end and four chips toward the other end, with an empty area in the center. If you see a ninth chip (or two) in the center, that is a register/buffer/controller chip for registered memory.
If that came out of one of those machines, you may have a server motherboard (albeit single-CPU).
JAFO last edited by
@Glen85 I see 4 full-tower cases there, that alone gives them 'some' value.
Look at the mother boards in those machines, if you see 4 or more memory slots & 2 or more PCIe slots there's potential, if any of them have any AMD socket3 motherboards... hey now.... 8core Piledrivers have dropped under 100bucks since the release of Ryzen(12 & 16 cores arent much more expensive and dropping like a rock)...
Sorry for the silence, folks. I've not been well, I'm afraid. I have tons of mental health quirks going on, as yet, undiagnosed (the NHS is pretty dire with mental health, I've been waiting since 2006, for a proper diagnosis!).
I'm still not quite on form enough to turf my flat upside down to get all the machines out, but as I've stressed, there's no rush for this stuff, as I can't afford to do anything about it for quite a while yet.
A quick update though: I think I have a mixture of Pentium 4 and Celeron D CPUs in the machines.
When I'm up to it, I'll get them all out and take loads of photos, giving all the information I can on everything. I've lost some of the RAM though, so goodness knows.
Please don't continue the conversation until I've listed everything, I won't be able to take stuff in and I don't want to upset anyone by not responding properly or forgetting to mention anyone.
Thanks for all your help and patience.