Milestones and Ubuntu/Debian Developer Packages

So it's been a productive weekend, besides the architecture doc/diagram I put up this morning there is now a decent list of milestones and features is now available leading up to the initial planned release early December. Here's the deal, my wife is expecting our third (and final!) child in mid-December so all bet's are off after that. So I've got quite an incentive get some of this done. Of more practical value a tarball containing Ubuntu/Debian packages for JXTA, JRuby, and the environment scripts is now up on Sluggish Sourceforge. No code is available, but this sets up all the dependencies described in the dev environment wiki page.

Rough Architecture Doc/Diagram on Wiki

I've put up a work in progress architecture document with a breakdown of PeerTab node components and functions that elaborates on this diagram.

But here is the high level overview of a minimal (edge) PeerTab node:

1. Various event sources (which could be local or remote, for example the node could be a syslog-ng server receiving a variety of syslog-style events or a netflow collector using flow-tools) are saved locally on the filesystem so that
   
2. Based on filtering criteria a subset of this information is added as events in a local database on the node which can be
3. Searched locally via the cli or remotely via the JXTA network via other nodes that are part of the same network
   

Bejtlich on Anemone and End System Monitoring

Bejtlich's Blog on Anemone reminded me on the [under]value of tools like iplog (it seems to have disappeared from the Debian package repository, it must have been back in 3.0 days and I know there were 3-4 of these types of tools, portsentry was another one) that simply log 3-5 tuple (src/dst/proto and ports) are raved about when they come from a router (i.e. NetFlow) but this value of this on the end system never got the attention it deserved? Maybe because there was no decent way to do analysis? Or the sort of logging was integrated into host-firewalls or IPS? Not sure.

Anyway, as part of more normal new Debian workstation/laptop install, I would always enable udp/tcp/icmp logging and back in the wild west worm days (2001-2003?) there was pretty interesting stuff deep within the campus network of a large enterprise where I worked. And before I knew what a BBMD was I found it due to the UDP broadcasts to 47808.

So the belief that there should be an easy for "dumb" end-hosts (that may be constantly joining, leaving, moving from wired to wireless, etc.) to share arbitray security info and P2P architectures such as those provided by JXTA seem to be a logical means of doing so. Whether something like this ultimately proves to be deployable or even useful, we shall see, but there is a lot more development ahead before we can say.

(BTW: the diagram came from Chapter 4 of the JXSE 2.5 Programmers guide)

First Glorious Screencast!

I could be wrong, but this is probably the only JRUBY + JXTA screencast out there. Perhaps the only JXTA screencast?

Using Wink, a free screen capture tool for Linux or Windows, I threw together a quick demo of where I'm headed in terms of of developing a JRuby abstraction layer of insulating developers from the gory details of JXTA. Nothing fancy, I basically set some environment variables, fire up jirb and create a new JXTA peergroup in about two lines of ruby.

And the best thing is you don't get to hear my grumpy, allergy-laden voice!

Setting up a PeerTAB Dev Environment

I created wiki page details the following steps necessary to set up your own peertab development environment, including:

• OS Installation (I used Debian Etch but it doesn't matter) and additional packages
  
• JRE/JDK (again I'm using the Java 5 packages
• Retrieving the source tree via subversion
• Installing JRuby
• Installing JXTA
• Running a simple hello world script I wrote several months ago
• Configuring fprobe and flow-tools (NetFlow will be the first EventSource) to be added to the network
  

Why JXTA? Updated JXTA Developer Documentation!

Back in May I started looking at what Open Source P2P APIs were available. P2PS looked promising (and even still under active development) since JXTA is probably overkill for this project, but there simply wasn't a critical mass of developers/developmentr compared to what was available for JXTA. As a result there weren't enough sample apps to look or solid programmer documentation. Free Pastry was also a possibility and had more sample apps, but didn't have a rich set of security functionality. Of course if the toolset is designed correctly, it should be easy to swap in different P2P APIs (for example, if the JXTA Ruby bindings become mature enough if we want to deploy embedded nodes that don't require Java)

I was pleasantly surprised when I visited JXSE Java.net Page today (JXSE is simply the Java SE/EE implementation, as opposed to CLDC/J2ME) I was pleased to see that a new version of the programmers guide was only released a couple of weeks ago. This is good news. JXSE is currently at verison 2.5_rc3 so given the early state we are at it makes sense to go with this.

Concept Document Available

Although nothing has actually been written on this document since June, I finally put up an overview/concept document I wrote (with the help of a few others, Gadi Evron in particular) to help define the problem space.

...it does not take much creativity to imagine a toolset that utilizes the types of networks used by Skype, Gnutella, or Bittorrent to provide a global view of Internet threat activity where users might choose to view, contribute, or search security information sources as easily as they would share a directory on their filesystem or download MP3’s. Initially, this network might only provide equivalent data as DShield (sources, protocols, and ports) with several obvious advantages made possible through its distributed architecture: direct (i.e. non-aggregated) access to collection nodes, the lack of a single data owner, flexible group membership (and policy definition) whereby organizations and collections of individuals to share information within an Enterprise across the Internet.

A PDF version is also available, unless you are too scared to read PDFs anymore.