| T O P I C R E V I E W |
| AndyThermco |
Posted - May 10 2024 : 15:33:31
Hi
I'm using a TImageEnFolderMView to load thumbnails of all images in a folder.
I have set: Grid->EnableLoadEXIFThumbnails = true; (C++)
but I can't find a way to tell it how to load RAW thumbnails if the EXIF thumbnail isn't available. I have the global settings to use ieenDLL as the raw engine.
As it stands, speed is OK when browsing an SSD. Nothing like as fast as Fast RAW Viewer, but adequate. However, when I try to browse an SD Card, with the idea of selecting images to pull onto the SSD, speed is impossibly slow. We're talking a minute or more to just show a screen of thumbnails, and a minute or more each time I scroll the page.
I have threads set to 4, caching to 10,000 and store type as ietFastThumb and rfNone for Thumbnail display filter.
I load the folder just by setting Folder property to the desired folder. If I change folder, I call Clear() on the control to stop any background threads already running.
Is there anything I can do to speed up the loading of thumbnails from RAW files? Canon CR3 is particularly slow...
Andy
Andy |
| 12 L A T E S T R E P L I E S (Newest First) |
| Andy_T_Bell |
Posted - May 14 2024 : 03:19:13
Nigel
TIEFolderTree is certainly better behaved than the TRzShellTree...
As I'm using C++ Builder I have attached an equivalent project in C+ Builder 12.1, with both 32 and 64 bit targets (the 64 bit app uses the new 'modern' 64 bit compiler, and I had to recompile the ImageEn code with this target to make a static library for it). The zip contains the compiled apps, but not the ImageEn DLLs, as these made the zip too large to upload. The DLLs are present when I run the apps, of copurse.
This is on my main dev PC, running ImageEn 12.2.0. The main dev PC is much faster than my laptop...
What I've noticed is:
The number of threads has a definite bearing - the sample app contains a box to set the number of threads, so I can change and assess it in realtime.
The type of image has a massive bearing: The C++ Builder apps have no issues with JPEGs - I have 1,000 JPEGs on the SD card and I can push the threads up to 6 without issues. 10 threads is too many and the app starts to stall.
With 1000 Canon CR3's on the SD card, setting the thread pool size to 1 seems to give the best performance when compiled at 64 bits. It's not fast, but it doesn't stall.
By 'performance' I mean, not just the loading of images into the grid, but also the ability to scroll up and down the grid.
The 32 bit app is a little faster with the RAW files, but still is best with just 1 thread.
If I move the images to my hard disk, performance is a little faster for RAW images. However, the threadpool size of 1 gives still gives best performance. In an ideal world, the UI should remain 100% responsive while the threads do their thing. With more than 2 threads in the threadpool, the application locks up if I try to scroll over a large number of RAW images..
If the images are on an internal SSD, performance improves massively.
Task Manager shows the app consumes about 5% CPU max, which isn't enough to explain the lockups.
Given all the investigations I've done, I think the issue is not with ImageEn's extracting the JPEG from the RAW file. My tests, like Nigel's, shows it does this as quickly as using LibRaw directly.
This issue seems to be with the UI <-> thread interaction. And when threads are taking longer to complete, as they would with many RAW images on SD cards, the UI locks up regularly and for extended periods.
When I faced this sort of thing in my own code I found several possible causes:
1) CPU overload due to too many threads (not the case here, I think)
2) UI being overwhelmed by threads sending too many PostMessage messages to it
3) Threads deadlocking/queuing because of synchronisation issues, including calling Synchronized methods in the main threads
4) The UI waiting from threads to complete rather than using a fast method of polling their status
I don't know the ImageEn code well enough to know if one or more of these could be causing the problem. If I were a g*mbling man, I would say 2) or 4) (or both) were the most likely. In themselves, blocked threads shouldn't affect the UI, unless somehow they are using the UI thread or it is waiting for them to complete/report progress...
I have excluded my application and the folders where the images are stored from being checked by anti-virus on my work PC. On my home laptop, where I have more control, I turned off all security to make sure nothing else was blocking the app. But the issues persist.
attach/Andy_T_Bell/20245142421_Browser.zip
9024.15 KB
Andy Bell |
| xequte |
Posted - May 14 2024 : 00:24:39
Hmm, I tried using a TTimer to force a refresh every 1/10 second (plus other intervals), but it still worked fine in our testing. Perhaps use TIEFolderTree instead. I have confirmed it does not trigger a TImageEnFolderMView refresh, even if AutoRefresh is enabled.
Nigel
Xequte Software
www.imageen.com
|
| xequte |
Posted - May 13 2024 : 23:44:48
Hi Andy
That might explain why I was having so much difficulty trying to reproduce the problem. As I don't have TRzShellTree I substituted it for TIEFolderTree. I will try introducing some timed refreshes to force a lockup and then hopefully we can implement a solution.
Nigel
Xequte Software
www.imageen.com
|
| AndyThermco |
Posted - May 13 2024 : 13:05:21
I applied a change to stop the Shell Tree updating. On my desktop, which is full on dev PC, the performance was better.
On my gaming laptop, browsing an SD card with 1000 CR3s on it, the app locked up again... This time Madexcept shows issues with the TIECustomMView, particularly painting the window.
This is using the simple Delphi app posted above, with the Options for the TRzShellTree having StDynamicRefresh set to false.
The MadExcept report is attached - this is for a 64 bit Delphi application, ImageEn 13.0.0.
I suspect, but could be wrong (again) that there are too many 'redraw' messages being received by the main thread, causing it lock up...
attach/AndyThermco/20245131353_bugreport.txt
Sorry - I also seem to have accounts here... Not intentional... |
| Andy_T_Bell |
Posted - May 13 2024 : 05:40:09
Nigel
After further testing this morning I have established the following, and I think it's good news:
The problem seems limited to 64 bit applications. When built as 32 bit, the application performed very well.
I switched back to 64 bit and enabled MadExcept and told it to detect a frozen main thread. It duly obliged and has logged a number of freezes.
The MadExcept log showed that the TRzShellTree component was in the heart of the UI freeze's stack trace.
I think it was trying to refesh itself each time it detected a change on the disk, and the adding of images to the Cache was sufficient for if to trigger a refresh. As images are being added to the cache rapidly, it caused it to go into spasms... It wasn't sending a 'Folder changed' event to my app, so I hadn't detected that it was refreshing itself.
If I turn off its Dynamic Refresh, the problem goes away...
This is only happening in 64-bits... It behaves better in 32 bits.
Sorry to have raised this thread, but at least it is now resolved...
Andy
Andy Bell |
| AndyThermco |
Posted - May 12 2024 : 06:51:42
Nigel
Thanks - I appreciate your help.
If I'm reading the code right (and Delphi is not one of my strengths) function IEReadCameraRAWStream_libraw in ieraw.pas reads the entire image into a memory stream and then decides what to do with it.
If that's the case, then this will cause speed issues when dealing with slow mediums, such as SD cards packed full of images.
The LibRaw functions mentioned above only read in the file's header to locate the position of the jpeg and then extract it.
If my assessment is correct, the ieraw.pas code, which of course has to cover every possibility, is doing too much in this particular instance...
EDIT
My tests show that by setting:
IEGlobalSettings()->CameraRawEngine = ieenDLL;
IE->IO->Params->RAW_GetExifThumbnail = true;
IE->IO->Params->RAW_GetEmbeddedJpeg = true;
IE->IO->Params->RAW_EmbeddedJpegLoading = ierlScaled;
IE->IO->Params->RAW_EmbeddedJpegMinWidth = 200;
IE->IO->Params->RAW_EmbeddedJpegMinHeight = 200;
The the usual ImageEnView->IO->LoadFromFile method is as fast as the LibRaw code..., so the lockup on loading many images is not the method of extracting RAW JPEGS...But this is over a single TImageEnView and the control in question is a TImageEnFolderMView.
I think the clue is that the main UI thread locks up. When stepping through the loading in my little Delphi sample project, I loads each image via TImageEnIO.SyncLoadFromStreamRAW
Does the 'Sync' part of the name indicate that this method is somehow synchronised with the main UI thread? If it does then this could cause the issues I'm seeing. If not, something else is hitting the main thread and causing the lockups.
Main clues, from my point of view, are:
The main UI thread locks up
The more threads, the more it locks up
The UI locks up more when loading images from a slow medium, such as an SD card
To me, that points to the threads loading the images doing too much work in the main thread. Using a fast internal SSD hides this issue, but slower disks expose it. |
| xequte |
Posted - May 11 2024 : 19:07:35
Hi Andy
I won't be able to test your code until tomorrow, but I did some performance testing of the various load options:
// Testing loading of 100 raw images of various types:
IEGlobalSettings().CameraRawEngine := ieenDLL;
ImageEnView1.IO.Params.RAW_GetExifThumbnail := loadExif;
ImageEnView1.IO.Params.RAW_GetEmbeddedJpeg := loadJpeg or loadExif;
ImageEnView1.IO.Params.RAW_EmbeddedJpegLoading := jpegMethod;
ImageEnView1.IO.Params.RAW_EmbeddedJpegMinWidth := 200;
ImageEnView1.IO.Params.RAW_EmbeddedJpegMinHeight := 200;
ImageEnView1.IO.Params.RAW_HalfSize := loadHalf or loadJpeg or loadExif;
t := GetTickCount;
for i := 0 to ss.Count - 1 do
ImageEnView1.IO.LoadFromFile( ss[i] );
memo1.Lines.Add('Load: ' + FloatToStr( GetTickCount - t / 1000 ));
RAW_GetExifThumbnail Loading: 1.657 ms
RAW_GetEmbeddedJpeg (Scaled) Loading: 1.734 ms
RAW_GetEmbeddedJpeg (WIC) Loading: 2.156 ms
RAW_GetEmbeddedJpeg (Default) Loading: 2.984 ms
RAW_HalfSize Loading: 8.625 ms
Normal Loading: 19.422 ms
Nigel
Xequte Software
www.imageen.com
|
| AndyThermco |
Posted - May 11 2024 : 08:36:44
attach/AndyThermco/202451181623_Browser.zip
74.71 KB
Attached is the simplest project I could think of to replicate the problem. It's a Delphi 12.1 64 bit app and ielib64.dll is present in the same folder as the executable file the project produces.
You may need to adjust the cache path to a different location.
With 2 threads in the thread pool, it operates OK when I navigate to folders with RAW images on my internal SSD - increasing the pool size to 4 speeds things up a bit. I would not describe it as rapid.
However, point this at an SSD card full of images (mine currently has 1,000 CR3's on it - nothing unusual after a photo trip) then when I select the folder:
1) A LONG delay, and the UI is frozen
2) The first screen of thumbnails appears
If I then start scrolling, the UI gets increasingly tied up in knots. Task Manager shows about 25% cpu activity for the app, so it is doing something. Eventually, the screen shows a page of thumbnails. If I page down several times it locks up more.
If I use Fast Raw Viewer, Photo Mechanic or DXO Photo Lab to access the SD card, the images display rapidly, with no lockups.
If you're able to tweak this demo to make it work faster, please let me know what settings to use. Otherwise, I think there's a problem with how TImageEnFolderMView is handling this scenario - even if the images take a long time to load, the UI shouldn't freeze if threads are being used.
ADD:
Using Libraw, I extracted all the embedded JPEGS and stored them in a separate folder on the SD card. I then pointed the test app at it and it browsed it acceptably quickly. So, it seems the bottleneck is in the Libraw thumbnail/embedded JPEG extraction. I'm no delphi programmer, but I stepped thru the code in ieraw.pas using the debugger and it is doing a lot that I didn't recognise... I know it needs to cover all bases, but the calls that Libraw say to use to get the embedded JPEG weren't there.
In C++, I do this to get the embedded JPEG (I've removed error checking to make this simpler):
LibRaw libRaw;
auto ok = libRaw.open_file(path.c_str());
auto res = libRaw.unpack_thumb();
auto thumb = libRaw.dcraw_make_mem_thumb(&res);
//can load into ImageEn if desired
IO->LoadFromBuffer(thumb->data, thumb->data_size, ioJPEG);
//can resize as needed, etc
//tidy up
libRaw.dcraw_clear_mem(thumb);
This is how I did it in an old app and it is fairly fast...
Andy |
| AndyThermco |
Posted - May 11 2024 : 04:40:48
I'm using ietFastThumb and a thumbnail size of 200x200 so, if I'm reading you correctly, I don't need to call OptimizeLoadingParams().
Interestingly, if I *reduce* the thread pool size, or even set it to zero, performance improves. Sounds counterintuitive, but I'm guessing that the threads communicate with TImageEnFolderMView using Windows messaging (PostMessage/SendMessage). If that is the case, then weird timing issues can occur. Basically, if Windows sends/posts messages faster than the app can respond to, everything jams up for a while - often causing the UI to freeze... Or maybe the threads call a synchronized method on the UI and that is jamming up?
Thinking back to when I used LibRaw, I was using multiple threads to extract the JPEG from the raw image, resize it, store it in a NexusDB and then use a TDBCtrlGrid to display them. There was a ton of event code to ensure that the threads only loaded images into the DB when the TDBCtrlGrid requested them, to try to make it as fast as possible.
However, at first, I ran into the exact same issues I am seeing now. Then I changed my code to simply use a timer on the UI to refresh the Nexus table every second or so if a thread had set a flag saying an image (or images) had been processed. The result was very fast with no lockups, despite the time spent storing the thumbnails in the DB. My current use of threads in C++ is to have them notify they are complete by storing their result in a vector (or a TList, etc) which the UI looks at every now and again. By using locks to guard the vector/list I avoid deadlocks and system overloads.
Of course, TImageEnFolderMView might be doing it using a similar mechanism, in which case there's something else causing the slowdown.
Add: I've looked at the ImageEn code and can see PostMessage used in the IO code in the context of threads. What is puzzling me is that this slowdown occurs when reading an SD card, where the threads take longer to extract the thumbnails, and I'd expect 'thread-messaging-overload' to be less. Also, the main UI thread literally locks up when browsing the SD card. As soon as I scroll past whatever thumbnails are already loaded it locks up, often the TImageEnFolderMView goes blank and Windows displays the 'app is not responding' message. I don't understand this...
Andy |
| xequte |
Posted - May 11 2024 : 03:05:02
Hi Andy
If your StoreType is not ietNormal then TImageEnFolderMView will do it for you automatically (which it should be, as the default is ietFastThumb):
http://www.imageen.com/help/TImageEnMView.StoreType.html
Using LibRaw directly should not be faster, as ImageEn should be using the same options, including loading the embedded JPEG. Please take a look at TIOParams.OptimizeLoadingParams() in iexBitmaps.pas. You will see that RAW_GetEmbeddedJpeg is always used if the image is bigger than your desired output size, RAW_HalfSize is used if output size is less than 1400x1400, and RAW_GetExifThumbnail is used if the output size is less than 150x100.
Nigel
Xequte Software
www.imageen.com
|
| AndyThermco |
Posted - May 11 2024 : 02:33:00
Thanks Nigel
But how do I do that for an TImageEnFolderMView?
I can't call it directly, because its MIO->Params property needs the image index. So I've added this line in the OnImageAdded event:
Grid->MIO->Params[idx]->OptimizeLoadingParams(wd, ht,true);
Is that the correct way to do it? Surely there is/should be a function to do this for the control as a whole?
In any case, this code does speed things up a bit, but it is still very slow when parsing an SD card with a lot of images.
I was hoping to avoid using LibRaw directly, as TImageEnFolderMView gives me almost everything I need, but I may have to resort to using LibRaw's ability to extract the embedded JPEG and point TImageEnFolderMView to a temporary folder containing them... I know from past experience that LibRaw's extraction of embedded JPEGS is much faster that what I'm achieving here.
Andy |
| xequte |
Posted - May 11 2024 : 01:49:37
Hi Andy
The best method is to call OptimizeLoadingParams() with the desired image dimensions:
http://www.imageen.com/help/TIOParams.OptimizeLoadingParams.html
It will enable all the available performance options, such as:
- RAW_GetExifThumbnail
- RAW_GetEmbeddedJpeg
- RAW_HalfSize
Nigel
Xequte Software
www.imageen.com
|
|
|
Search
