Ruby: RubyGems

RubyGems is a new approach to managing packages of Ruby code. The most convenient feature for the average user is that gems (apps and/or libs) can be downloaded and installed in one step from the command-line. For example, this command downloads and installs 'log4r'.

  gem install log4r

Any other gems that log4r depends on will be grabbed as well. Anyway, RubyGems has its own Wiki, which you can browse at HTTP://rubygems.rubyforge.org. It includes a brief overview as well as detailed information. The current status (2004-04-26) is still very much alpha, but very much working. Note: The living, breathing RubyGems has been in development since October 2003 or so. Prior to that, another project of the same name and the same goals was in operation, but it has petered out. The text below concerns the old RubyGems, and is left here for posterity.

For those wondering, I have returned from my Ruby hiatus and intend to lend whatever support I can to the new RubyGems project, including reviving some of my original concepts if they have been lost to the new developers. The original RubyGems petered out 2 years ago mostly because of the large amount of new code it required, much of which is now supplied in the default Ruby installation. With this new support, I think the new RubyGems can finally become what I originally envisioned. -- RyanLeavengood

  OLD RUBYGEMS MATERIAL BELOW!

RubyGems is a system that aims to provide the following (it is still in development):

A development framework that allows developers working on pure-Ruby projects to packages those projects into a single file that contains their source files as well as metadata describing the project. This metadata includes the name for the project, the author's name, a category, a version, a description, and even dependencies the project has.
- Also, locale metadata as in: lang=en, country=us, charset=us-ascii [vruz]
A user framework that allows users to easily use the single self-contained files created by developers in their own development or with any applications that they use. This framework transparently accesses the files contained within the gem files, and allows them to be loaded or required from other Ruby programs without a need for extraction or any complicated installation.

Other general features of the system include:

The ability to completely customize the operation of the RubyGems system through simple text-based user-specific properties files. Some possible properties include setting local user depositories for RubyGems (alleviating the need to be root on multi-user systems to try out new software), security options and other options for the RubyGems peer-to-peer network system (described below), and more.
A framework for automatically running any RubyUnit tests contained within a gem during its installation.
A standard form of class and module documentation (a la JavaDoc?) and multiple ways to access that documentation. This will likely use one of the current systems (e.g. rdtool or RubyDoc?), possibly in a modified form.
A peer-to-peer networking system that allows for:
- searching for gems based on any of the metadata contained within the gem files as well as contained source files.
- easy sharing of gems between trusted peers.
- validation of gem files using MD5 checksums downloaded from multiple trusted servers.
- automatic downloading and installation of gems.
- automatic releases and updating of new versions of gems.
- and whatever else I can think of :)
  - It would be nice if you could also do searches on locale, language, country, charset [vruz]

I expect to release RubyGems 0.5 in early October 2001. This version will only include the core features mentioned in the first two bullets above (developer tools to create gems files and user tools 1000 to read and use them.) --RyanLeavengood

''Sounds fantastic! I like the idea of one gem specifying that it requires another. I can imagine the peer-to-peer system installing one gem and then finding and installing the gems that it needs, and so on, without any extra intervention from the user. Could the metadata be extensible so that gems could be the basis for the component models of different frameworks? Sort of a COM layer for Ruby, through which higher-level frameworks could locate, load, instantiate and interconnect components based upon framework-specific metadata. E.g. a web browser could look for gems that display different MIME types, a naming service could look for gems that talk different directory protocols, and so on.'' --NatPryce

Yes, similar to the Debian package system I intend for the P2P system to automatically resolve dependencies between gems. Regarding extensible metadata, yes I think that is also a great idea. After some discussion with our friend PragDave, I have decided to support multiple file types (simple text and binary to start with), and in the process will also support extensible metadata in these file types. Everything you mention is very interesting and something I will definitely keep in mind as I continue developing RubyGems. --RyanLeavengood

''Have you considered how much of the P2P system is preestablished infrastructure and how much is configured ad-hoc? Ad-hoc p2p networks present a number of problem that are not easily solved. TMN, no adhoc p2p network has been proven stable and scalable in practice.''

You make a good point. My thoughts were to just leverage the P2P technologies to provide a common interface for services related to RubyGems. I never thought a purely ad hoc P2P network would be very good for providing consistant access to all the gem files that have been released (for the reasons you mention.) Instead I envisioned a ring of "Master Peers" (preestablished infrastructure I suppose) that would each contain all the released gem files. This allows for a more scalable and stable architecture that essentially has built-in load balancing and mirroring. Any additional peers that join up and choose to provide the RubyGems Content Provider Service (or whatever it is called), will just act as additional file sources that might allow for quicker downloads to certain other peers based on network topology or load, etc. Also, as I mention above, there is a need for "trusted servers" that can provide MD5 checksums for all the released RubyGems files. The "Master Peers" mentioned above could also provide this service. For security reasons, only peers in a certain peer group could provide these Master Peer security services, and all peers that choose to join this peer group would need to be validated and approved before they can join. These concepts of peer groups and peer group services have come from my studies into the JXTA (HTTP://www.jxta.org) framework (though I doubt that the concepts are unique to JXTA.) Of course given how young this technology is, it is hard to say whether any of the above will really work, or how secure the final system will be. At this point my plan is to look into prototyping a Ruby implementation of the basic JXTA protocols, and start experimenting. If it turns out that none of the above plans are feasible, a more standard client/server architecture could be designed and put on top of RubyGems, just as I plan with this P2P system. Nothing about the core RubyGems system makes either of these favorable over the other. I just think P2P is an interesting new technology that could produce a very advanced system for Ruby library distribution. --RyanLeavengood

This all sounds very interesting. Clearly progress could be made on both the "how gems are sourced" strategy and the other areas such as the meta-architecture, local install issues, etc, in parallel. RustyF

From the description above, it is un 4c3 clear (to me) if the system includes the CRITICAL feature of library versioning. Without library versioning, tracking dependencies does not work in practice - the system will be continually unstable. A version number can be based on the following parts (all of which should be set in a dependency request):

A major version numbers. Any change that remove from the exported API needs to change this, as the removal means that packages written to the old API cannot be guaranteed to work. The major version number is really part of the name of the package, but people get attached to names... (This should be coded into the require lines, too, so it really should be part of the name.)
A minor version number. Any change that add functionality needs to bump this. When a package requires another package, you need to get the same or higher minor number. You may choose to bump or not bump this on bugfix releases, depending on how important the bug seems.

Without versioning support, you'll get breakage in applications (and other libraries) whenever somebody update a library. (You should also note that just about any application might be used as a library, so...) --EivindEklund

There is a RubyGemsToDoList for people to beat into shape to assist progress towards integration into Ruby 1.8.4.

oh my god, an other packageing system.. why not simply use dpkg or rpm?

: If they are available for your system, go ahead and use them. If not, then gems is a good option.

RubyGems (Ruby)