Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization needed: SoilFileLockRegistry>>#addLock: slow for many locks #632

Open
MarcusDenker opened this issue Feb 20, 2024 · 2 comments

Comments

@MarcusDenker
Copy link
Contributor

MarcusDenker commented Feb 20, 2024

this example stores 130K strings (alltogether ~40MB) in an indexed dictionary.

(This is therefore not critical as this is not something happening in the use cases we target now, but should be fixed at some point)

soil := Soil path: 'soil-bench'.
	soil 
		destroy;
		initializeFilesystem.
dict := SoilBTreeDictionary  new
		keySize: 4;
		maxLevel: 8; "ignored for BTree"
		yourself.
		
tx := soil newTransaction.
tx root: dict.

Smalltalk globals allMethods do: [ :meth |
	dict at: meth sourcePointer put: meth sourceCode.
	].
tx commit.

The commit is very slow. Time is spend in SoilFileLockRegistry>>#addLock: seemingly

I had to comment out the already locked check to make it work. Doing a query works nicely after is is commited:

tx := soil newTransaction.
(tx root at: (OrderedCollection>>#do:) sourcePointer) soilRealObject

Problem:

The locks are in an OrderedCollection, checking them gets too slow when there are many (the example creates 130K locks).

We need a DataStucture that allows us to check interval overlap fast, e.g.

https://en.wikipedia.org/wiki/Interval_tree

@MarcusDenker
Copy link
Contributor Author

Splitting the commits at a class boundary is much faster (25 seconds):

Smalltalk globals allClasses do: [ :cl  | | tr |
		tr := soil newTransaction.
		cl localMethods do:
			[ :meth |
			tr root at: meth sourcePointer put: meth sourceCode].
	tr commit.
]

@pdebruic
Copy link

Years ago I made an implementation of a Centered Interval Tree in pharo:

https://github.com/pdebruic/centered-interval-tree

I don't remember it being trash but it might be ;)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

No branches or pull requests

2 participants