OFBiz › OFBiz - Dev

[jira] Created: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

_‹ Previous Topic Next Topic _›

Classic

List

Threaded

15 messages Options

Nicolas Malin (Jira)

[jira] Created: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

Current implementation of UtilCache has a memory leak
-----------------------------------------------------

Key: OFBIZ-4220
URL: https://issues.apache.org/jira/browse/OFBIZ-4220
Project: OFBiz
Issue Type: Bug
Components: framework
Affects Versions: SVN trunk
Environment: ALL
Reporter: Philippe Mouawad

Hello,
Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();

I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.

The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.

We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Updated: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
> Attachments: patch_OFBIZ-4220.patch
>
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Updated: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Updated: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Commented: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008845#comment-13008845 ]

Philippe Mouawad commented on OFBIZ-4220:
-----------------------------------------

There is another source for this leak in:
ExecutionPool registration.

Object retains lot of references to Caches that may be not be used anymore.

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Updated: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java
>
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Issue Comment Edited: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008845#comment-13008845 ]

Philippe Mouawad edited comment on OFBIZ-4220 at 3/20/11 3:54 PM:
------------------------------------------------------------------

The main source for this leak is:
ExecutionPool registration.

Object retains lot of references to Caches that may be not be used anymore.
I made a test with 256m affected , and ExecutionPool retains through 164 Mo in an array of 1134867 elements.
I made a new test by commenting ExecutionPool.addPulse(line); in tryRegister and no OOM occurs.

From my understanding:
- ExecutionPool is used to collect elements in the Cache and handle expiry
- ExecutionPoolPulseWorker runs in N threads to cleanup the queue , where N is the number of processors

if I understand well I think issue comes from the fact that removed elements are still in Expiry Queue and in the test case PUT is too heavy (50 threads) while expiry threads are only 2.
But this test case mimics what may happen in real life.

Philippe
http://www.ubik-ingenierie.com

was (Author: [hidden email]):
There is another source for this leak in:
ExecutionPool registration.

Object retains lot of references to Caches that may be not be used anymore.

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java
>
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] Updated: (OFBIZ-4220) Current implementation of UtilCache has a memory leak

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philippe Mouawad updated OFBIZ-4220:
------------------------------------

Attachment: patch-4220.patch

Fix to the ExecutionPool leak.

You can test its efficiency with provided TestMemoryLeak
Philippe Mouawad
http://www.ubik-ingenierie.com

> Current implementation of UtilCache has a memory leak
> -----------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java, leak.png, patch-4220.patch
>
>
> Hello,
> Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
> private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();
> I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
> Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.
> The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.
> We made some load testing and current implementation ends with an OutOfMemory.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Updated] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Philippe Mouawad updated OFBIZ-4220:
------------------------------------

Priority: Critical (was: Major)
Description:
If maxInMemory is set on caches, then current implementation will leak if many puts are done in cache.
We made some load testing and current implementation ends very rapidly in an OutOfMemory.

Philippe Mouawad
http://www.ubik-ingenierie.com

was:
Hello,
Since some time UtilCache has been updated to replace the WeakHashMap by a ConcurrentHashMap:
private static final ConcurrentHashMap<String, UtilCache<?, ?>> utilCacheTable = new ConcurrentHashMap<String, UtilCache<?, ?>>();

I think this fix to concurrency issues has introduced a memory leak or at least a much bigger impact on Memory.
Indeed descendent of AbstractCache (particularly AbstractEntityConditionCache, EntityListCache) tend to create many caches that do not need to always persist in time.

The old WeakHashMap handled this by enabling them to disappear while today the Hard ref disables this.

We made some load testing and current implementation ends with an OutOfMemory.

Summary: Current implementation of UtilCache has a memory leak if maxInMemory is set (was: Current implementation of UtilCache has a memory leak)

> Current implementation of UtilCache has a memory leak if maxInMemory is set
> ---------------------------------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Priority: Critical
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java, leak.png, patch-4220.patch
>
>
> If maxInMemory is set on caches, then current implementation will leak if many puts are done in cache.
> We made some load testing and current implementation ends very rapidly in an OutOfMemory.
> Philippe Mouawad
> http://www.ubik-ingenierie.com

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Commented] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009075#comment-13009075 ]

Jacques Le Roux commented on OFBIZ-4220:
----------------------------------------

Thanks Philippe for the detailed report!

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Commented] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009164#comment-13009164 ]

Erwan de FERRIERES commented on OFBIZ-4220:
-------------------------------------------

Hi Philippe,

I just saw there is also a newer lib version at http://code.google.com/p/concurrentlinkedhashmap/.
Is there any point in updating too ?

Change log:
This version provides incremental improvements based on user feedback.

* Fixes issue 20 , which identified a race condition when updating a value's weight.
* Reduces memory usage by optimizing the buffering of recency operations.
* Reduces read contention by improving how a recency buffer is selected.
* Strict LRU reordering by a more intelligent draining algorithm.
* Support for limiting the capacity through a plug-in strategy.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Commented] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009332#comment-13009332 ]

Philippe Mouawad commented on OFBIZ-4220:
-----------------------------------------

Hello Erwan,
I cannot answer to this question, I didn't follow very closely this library.
If JUnit tests cover widely the class then there is no big risk , otherwise it would require more thorough testing.
I can test with this last version if you want but it will only answer to the leak issue.

Anyway issue is fixed even without upgrading.

Regards
Philippe

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Assigned] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erwan de FERRIERES reassigned OFBIZ-4220:
-----------------------------------------

Assignee: Erwan de FERRIERES

> Current implementation of UtilCache has a memory leak if maxInMemory is set
> ---------------------------------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Assignee: Erwan de FERRIERES
> Priority: Critical
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java, leak.png, patch-4220.patch
>
>
> If maxInMemory is set on caches, then current implementation will leak if many puts are done in cache.
> We made some load testing and current implementation ends very rapidly in an OutOfMemory.
> Philippe Mouawad
> http://www.ubik-ingenierie.com

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Commented] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13009606#comment-13009606 ]

Erwan de FERRIERES commented on OFBIZ-4220:
-------------------------------------------

No problem. I will work with your patch directly. If later you think this would need a lib migration you tell me.

Anyway, if a migration is needed or benefits the project, maybe switching to guava http://code.google.com/p/guava-libraries/ should be a better idea, as the 1.1 version is integrated in it.

cheers,

> Current implementation of UtilCache has a memory leak if maxInMemory is set
> ---------------------------------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Assignee: Erwan de FERRIERES
> Priority: Critical
> Labels: CACHE, LEAK, MEMORY
> Attachments: TestMemoryLeak.java, leak.png, patch-4220.patch
>
>
> If maxInMemory is set on caches, then current implementation will leak if many puts are done in cache.
> We made some load testing and current implementation ends very rapidly in an OutOfMemory.
> Philippe Mouawad
> http://www.ubik-ingenierie.com

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Nicolas Malin (Jira)

[jira] [Closed] (OFBIZ-4220) Current implementation of UtilCache has a memory leak if maxInMemory is set

In reply to this post by Nicolas Malin (Jira)

[ https://issues.apache.org/jira/browse/OFBIZ-4220?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Erwan de FERRIERES closed OFBIZ-4220.
-------------------------------------

> Current implementation of UtilCache has a memory leak if maxInMemory is set
> ---------------------------------------------------------------------------
>
> Key: OFBIZ-4220
> URL: https://issues.apache.org/jira/browse/OFBIZ-4220
> Project: OFBiz
> Issue Type: Bug
> Components: framework
> Affects Versions: SVN trunk
> Environment: ALL
> Reporter: Philippe Mouawad
> Assignee: Erwan de FERRIERES
> Priority: Critical
> Labels: CACHE, LEAK, MEMORY
> Fix For: SVN trunk
>
> Attachments: TestMemoryLeak.java, leak.png, patch-4220.patch
>
>
> If maxInMemory is set on caches, then current implementation will leak if many puts are done in cache.
> We made some load testing and current implementation ends very rapidly in an OutOfMemory.
> Philippe Mouawad
> http://www.ubik-ingenierie.com

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira