“Out of memory” error below memory limit

memory
multiview
fiji
macro
Tags: #<Tag:0x00007fb87c22ff38> #<Tag:0x00007fb87c22f718> #<Tag:0x00007fb87c22ec00> #<Tag:0x00007fb87c22e520>

#1

I’m working with the Multiview reconstruction plugin for deconvolution of lightsheet microscopy data. My error comes when I run the “deconvolve dataset” function repeatedly through a macro. This results in an “Out of Memory” error after around 38 repetitions of the function.

I have increased the maximum memory available to 180 Gb (Via the memory and threads option.) and run the garbage collector multiple times after the completion of each step in the macro. This results in the memory usage topping out at around 100 Gb each iteration, as displayed by the system monitor and imageJ memory monitor.

However I still receive “Out of Memory” errors. Does anyone have any idea where this memory limit could be coming from?

Edit: Full error message,
Out of memory
All available memory (1997MB) has been
used. To make more available, use the
Edit>Options>Memory & Threads command.

The Memory & Threads option is set to 180000 MB, so I am very confused.


#2

Hi @EdwardMuir

How much physical memory is available on your machine?

It is not a good idea to increase above 3/4 of the physical memory available
Further information can be found here.

If you put it to 180GB you would need around 240GB physical memory :wink:


#3

Thanks for the reply @tibuch.

I have 188 GB available, I’ll set the cap to 130 GB and report back.


#4

Repeating my macro with the memory limit set to 130 GB reduced the run time before crash to 28 repetitions. The memory usage, as tracked by the system monitor, still does not increase above 100 GB.


#5

I am still confused by this message. @ctrueden do you have any idea?

Are you using a CUDA implementation? If so, it could be an open issue.


#6

The confusing message is a bug in ImageJ 1.x. The error in question is generated here, using Runtime.totalMemory(), when the call should probably be Runtime.maxMemory() instead. The totalMemory() call returns “the total amount of memory currently available for current and future objects, measured in bytes” whereas maxMemory() reports “the maximum amount of memory that the virtual machine will attempt to use, measured in bytes.” I believe the intent was to report on the latter here, but the Runtime method names are a bit confusing—I personally have also written code in the past which had the same issue.

Anyway, it is nonetheless an issue with insufficient RAM. See also this Troubleshooting section, although probably not super helpful in this specific case.

I do think @EdwardMuir should be able to go higher than 130 GB… maybe 170GB? That still leaves 18GB for non-ImageJ stuff, which seems like plenty. The 3/4 rule breaks down when you have such a huge amount of RAM.

The next step to diagnose and debug would be to run ImageJ with JVisualVM or some such, to see which objects are taking up all the space.


#7

I believe this was fixed in the latest ImageJ 1.51k21 daily build:

Thanks @Wayne.


#8

Thanks for the response.

170 GB also ran for less iterations than 180 GB.

I will attempt to run ImageJ with JVisualVM and then report back.


#9

I have run the macro with a visualvm tracer running. However I don’t know how to interpret the results.

How do I check which “objects” are taking up space?

Attached is a screenshot of the memory and cpu usage over the run time of the macro/ plugin.


#10

@EdwardMuir Thanks for giving it a shot.

You want to perform memory profiling. See the “Memory Profiling” section of the Profiling Applications guide. Click the Profiler tab, then the Memory button.

Maybe @StephanPreibisch can help you interpret the results…