Hi, i spoke again with developer doug about this problem but he cant find a solution so far.
He will setup a 3 card system to find the problem and he is tough.
He ask if you could compile blender by yourself on your system, then he could send patches to you to test.
It can be hard to setup a build system if you never get in touch with software compiling and so forth.
I´ve read your PM.
Cheers, mib.
Well. I have compiled some code in C++, it will be nice to show me through TeamViewer how to do it, and I can handle it then.
Other strange thing that came out from my todays test. Utilizing the GPUs in 3D, Rendered viewport shading, in CYCLES. It seems in viewport it uses all GPUs, in comparison to Render. The loads of GPUs are unequally underutilized (probably because of progressive refine of whole viewport) and loaded as follow.
Rendering Mike Pan’s BMW scene in official 2.68 RC1. GPU, Load an Memory usage, measured using GPU-Z
In rendering inside viewport:
580 - 77-83% - 792MB
590 - 11-13% - 446MB
590 - 37-43% - 446MB
590 - 54-63% - 446MB
590 - 56-67% - 446MB
While in F12 rendering with 256x256 tiles
580 - up to 99% - 675MB
590 - 0% - 207MB
590 - up to 99% - 426MB
590 - up to 99% - 426MB
590 - up to 99% - 426MB
If you are on linux it is no problem to compile but on window i have no idea.
Cheers, mib.
EDIT: these are very interesting news, make it easier for the dev to find bug.
Put it in your bug report!
It Is in bug report now.
I am waiting for a new hard drive, I will have it probably tomorrow,
than I can make an Ubuntu installation on it, and we can get into the compiling 
Great job with 2.68, the improvements in Cycles is really awesome.
I am really curious, how we will make it even faster 
Hi guys, so the drive is in, linux installed Ubuntu 13.04, NVidia drivers 304.
I had pretty hard times with unstability of fresh install, untill i installed the drivers exactly this way:
now it is stable, running Blender 2.68, and the GPU problem is the same
Blender finds all 5GPUs, but renders tiles only with 4 of them
Same applies in case of 3x NVidia GeForce 590, Cycles utilizes only 4GPUs out of 6GPUs.
PS: When hit render in Octane, it shuts down my computer.
Wow 3 gtx590! I just read online that these cards suck around 500 watts under load. I think your 1500w enermax psu can’t handel three of these cards plus the rest of the system, thats why your PC shuts down.
I was surprised, that it is running on full load of all GPUs in Arion, in range of 1000-1050 watts. The Enermax 1500W Platimax PSU can handle 1650 peak for 60seconds. I am very sure it would even handle the 4x GTX 590.
You can use a Cooler Master PSU calculator http://www.coolermaster.outervision.com/ to estimate power consumption of a whole system. The shut down is probably caused by something else. I will test Octane later under Linux, to prove/disprove your assumption.
But you are true, I looked after the consumption and it can peak to 491W on one card … so I will make some testing, and then I drop another post.
EDIT: It turns off Octane in Linux too 
Well I think that Octane is probably using the gpus better so it needs also more power than Arion.
I don’t think that these online calculators are that precise, they probably differ much between different calculators.
I gooled for “dual psu 24 pin adapter cable” you can get such a thing for third of add2psu price, I found some schemes and will do one by myself. Then I will add my old 650W PSU, and do the same test then to see whether the reason was really the PSU. But it is a good item for beginners, thanks for the link.
Indeed. Difference in power consumption of 580+2x590, between Octane and Arion is 130W. This difference causes 3x590 system to shut off on 1500W PSU. We will see how it will come out in Cycles and power consumption of 6 GPUs after the 4GPU limit bugfix.
EDIT:
so the peak of 3x590 is about 1500W, plus the rest of the system that is about 200-250W, is imposible to manage at peak even if it can handle 1650W for a minute. So no way to have 3x590, nor 4x590 inside on 1500W as I taught.
I was estimating the power consumption of the whole HW setup based on NVidias official specifications of GTX 590 (Maximum Graphics Card Power - 365W) and Cooler Master PSU calculator which came out as a big mistake.
Thanks OscarM, for pointing me out on the 500W peak of 590s, I have got new experience out of this lesson :).
zajooo I uploaded my own blender build (link deleted)
Extract it and open a terminal there. I recommend Linuxmint15 (ubuntu 13.04 is a pain!) and we do right click and “open terminal” but in ubuntu you probably will have to do:
cd /folderwithblender
Well, once you are in a terminal inside the folder run blender using this order:
./blender
Now blender probably will not run and complain about libraries not present. So tell me and I will tell you, or just install them if you know.
If it runs do a render in GPU mode and tell me what is the message like this one below, that appears in your system (you can see in my system I have only 1 logical device and num is 1 too):
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
TileManager::gen_tiles_global() num: 1 num_logical_devices: 1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
(you can copy from the terminal selecting and right click and choose copy and then paste here. Ctr+C doesn’t work in terminal)
./blender: error while loading shared libraries: libIlmImf.so: cannot open shared object file: No such file or directory
EDIT: managed to install it with
libopenexr-dev_1.6.1-7_amd64.deb
I have setted up the scene to use cycles, all CUDA devices 580+590(4x), GPU Compute and the result is …
ubuntu@quad:~/Downloads/blender26$ ./blender
Color management: using fallback mode for management
Read new prefs: /home/ubuntu/.config/blender/2.68/config/userpref.blend
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
TileManager::gen_tiles_global() num: 1 num_logical_devices: 1
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
EDIT: and renders with 4 tiles
Well, seems I placed the code in a not correct place then. I was expecting to see num to have value 4. But 1 indicates that is not the place where the job for each card is created.
Let me think and tomorrow will post again.
New build (link deleted) to test.
Extract, enter in the folder in a terminal and:
./blender
and render using F12 and post the message it says (same formating as before so it is easily recognizable). If no message appears then make the F12 render in a floating window (Render panel, Display: New window).
ubuntu@quad:~/Downloads/blender26$ ./blender
Color management: using fallback mode for management
Read new prefs: /home/ubuntu/.config/blender/2.68/config/userpref.blend
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
device_multi.cpp task_add() devices.size: 5 tasks.size: 5
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
rendering with 4 tiles, in F12 and in Render panel, Display: New window F12 also, and shows the same
EDIT: do you thing it can have something to do with my Q6600, wich is a quadcore with no hyperthread ?
