Talk with Avi Bryant and Bob Walker
5/30/2008, 3:40PM, Portland, OR
Here are some of my notes from Railsconf 2008. Warning these notes are only vaguely edited (but arguably still useful). There are certainly typos and errors. If executing code from one of these articles, your mileage may vary (from spontaneous self destruction of everything you know and love to spontaneous coolness). Also, you may want to double check “facts”. You definitely should not use these articles as a spelling reference. Feel free to post corrections.
MagLev is rails that go fast. Typically have mongrel cluster talking to storage device and memcache instance for each server. Maglev unifies this stack: Storage, Cache, and Ruby
“Ordinary ruby shells”. Two terminals using the same shared objects! You can do this for any number of boxes across the entire system.
Local state is not shared, global variables ($hat).
Ruby shell has transactions as you are used them. Transactions are there for mutable state.Commits don’t only go to in memory caching, they go to disk and if there is a failure then the machines can be rebooted with the existing state. Could this be on multiple machines? Yes. The storage can be greater than ram because they can be persisted to disk.
The system does not currently run rails, but it will be soon. Can currently run webrick with hello world.
Question: Is the ruby spec implemented and at what level?
Next phase is implementing the ruby specs against this.
Question: What happens to files and sockets?
Each VM will have it’s own seperate copy of that resource and they will be nil’ed out when brought over.
Question: Is this a written from scratch ruby machine?
Yes, it is not MRI.
Gemstone founded in 1982 in Portland.
- Cretated GemStone smalltalk
- Enterprise proven technology
- Production systems deployed worldwide
- Support domains that are not well served by relational database systems
- Worldwide container
- 1 dual core up to really large systems
- Mutliple ruby virtual machines
- Oject Caches/Repository
They are currently 100 days into the implementation process.
- Dynamic Language VM
- Based on Gemsone/s 64VM
- Well tuned garbage collection
- Extended with new byte codes for Ruby
- JIT native code generation
- Shared caching uses shared memory
- Repository objects are read into the cache
- Fully recoverable ACID Transactions
- Simple API
- Conflict management
- Native Language
- Simple API
- You can store a trillion instances
- Up to 17PB
- bm_so_exceptions 5.6 x faster than MRI
- fib 12.5 x MRI
- vm2 method 24 x MRI
- bm_so_sieve 68.4 times
- bm_loop_whileloop2 111.5 x MRI
“Holy Shit” -Antonio Gangiano (Great Ruby Shootout)
Numbers will be independently verified shortly.
Audience question. What is missing from Ruby that will decrease this performance?
The plan is ruby spec and core library.
But does it scale?
What’s the vision?
Rails – taking it to a different level of scalability.
- Open Community Involvement
- Ruby Spec Compliance
- MagLev Ruby open source
- Free version of MagLev will be feature limited for small to medium scale
- Info @ maglev.gemstone.come
A lot of rails makes use of the relational database structure and indices. Will Maglev address these?
Yes. And you will be able to make your queries in the native language. Will likely clone the API but run gemstone underneath.
Yes. They are supported in smalltalk.
How to optomize? Will it be a linear scan?
Gemstone does will do an linear scan for a basic select but gemstone does have various indexing.
Will the Gemstone VM be open sourced (asked by Charles Nutter from jRuby team):
Bob: “In a world of infinite possibilities it’s possible.”
Will existing extensions in C be possible to integrate?
Not currently. Rubinius has done some good work in this area and it may be possible to use a similar approach for Gemstone. C extensions may not be necisary in certain cases.
However rails depends on C like BigDecimal(Wilson)?
Will solve those problems when they get to them.
ActiveRecord uses SQL fragments. How would that work since it’s not part of ruby?
To port a rails app may require getting rid of optomizations done in sql…. this is an area where feedback will be welcome. This is yet to be designed built.
Which pieces are open source?
All of the stuff at the ruby level is hopefully to be shared with Rubinius. ActiveRecord API will be. But as for the underlying VM Bob answered that earlier (e.g. “in a world of infinite possiblities….”).
Is the persistent store distributable?
More like typical SQL statement. Master slave replication…
How is the VM implemented?
In short… in C.
When is going to be ready?
Pay attention to what we are doing. More info in September.