PeerTAB: a P2P Threat Analysis Bus

Blew the First Milestone (of course) but some API Progress

2007-10-07T18:51:00.001-07:00

Various personal and professional obstacles kept me from being where I wanted to be tonight (doesn't that sound familiar) but I actually made some decent process on the API.

Here's what you can do so far:

Load and save JXTA configuration with a wrapper around NetworkConfigurator
Start, stop, create an EDGE node session with a wrapper around NetworkManager

Amazing, huh?

An excerpt is below


franz-g4:~/dev/playin/peertab/peertab/unstable mdfranz$ ./tncfg.rb
Reading config file from /Users/mdfranz/Documents/dev/playin/peertab/peertab/unstable/.jxta
Peer Name> bobo
Username> whatever
Password> secret
Save config?y
Oct 7, 2007 8:47:48 PM net.jxta.impl.membership.pse.PSEUtils 
INFO: Loaded Security Providers into system class loader
franz-g4:~/dev/playin/peertab/peertab/unstable mdfranz$ jirb
irb(main):001:0> require 'tabnet'
=> true
irb(main):002:0> s = TABNET::EdgeSession.new
irb(main):008:0> s.start
Oct 7, 2007 8:49:23 PM net.jxta.impl.membership.pse.PSEUtils 
INFO: Loaded Security Providers into system class loader
Oct 7, 2007 8:49:23 PM net.jxta.platform.NetworkManager configure
INFO: Loading existing configuration. mode = EDGE
Oct 7, 2007 8:49:23 PM net.jxta.platform.NetworkManager startNetwork
INFO: Starting JXTA Network! MODE = EDGE,  HOME = file:/Users/mdfranz/Documents/dev/playin/peertab/peertab/unstable/.jxta/
Oct 7, 2007 8:49:24 PM net.jxta.peergroup.WorldPeerGroupFactory newWorldPeerGroup
INFO: Making a new World Peer Group instance using : net.jxta.impl.peergroup.Platform
Oct 7, 2007 8:49:24 PM net.jxta.impl.cm.SrdiIndex clearSrdi
INFO: Clearing SRDI for null

If you want to see the whoe output see http://www.threatmind.net/secwiki/PeerTab/FakeCasts/1 because Blogger barfed on the output.

Milestones and Ubuntu/Debian Developer Packages

2007-09-30T18:52:00.000-07:00

So it's been a productive weekend, besides the architecture doc/diagram I put up this morning there is now a decent list of milestones and features is now available leading up to the initial planned release early December. Here's the deal, my wife is expecting our third (and final!) child in mid-December so all bet's are off after that. So I've got quite an incentive get some of this done. Of more practical value a tarball containing Ubuntu/Debian packages for JXTA, JRuby, and the environment scripts is now up on Sluggish Sourceforge. No code is available, but this sets up all the dependencies described in the dev environment wiki page.

Rough Architecture Doc/Diagram on Wiki

2007-09-30T08:25:00.000-07:00

I've put up a work in progress architecture document with a breakdown of PeerTab node components and functions that elaborates on this diagram.

But here is the high level overview of a minimal (edge) PeerTab node:

Various event sources (which could be local or remote, for example the node could be a syslog-ng server receiving a variety of syslog-style events or a netflow collector using flow-tools) are saved locally on the filesystem so that
Based on filtering criteria a subset of this information is added as events in a local database on the node which can be
Searched locally via the cli or remotely via the JXTA network via other nodes that are part of the same network

Bejtlich on Anemone and End System Monitoring

2007-09-29T09:00:00.000-07:00

Bejtlich's Blog on Anemone reminded me on the [under]value of tools like iplog (it seems to have disappeared from the Debian package repository, it must have been back in 3.0 days and I know there were 3-4 of these types of tools, portsentry was another one) that simply log 3-5 tuple (src/dst/proto and ports) are raved about when they come from a router (i.e. NetFlow) but this value of this on the end system never got the attention it deserved? Maybe because there was no decent way to do analysis? Or the sort of logging was integrated into host-firewalls or IPS? Not sure.

Anyway, as part of more normal new Debian workstation/laptop install, I would always enable udp/tcp/icmp logging and back in the wild west worm days (2001-2003?) there was pretty interesting stuff deep within the campus network of a large enterprise where I worked. And before I knew what a BBMD was I found it due to the UDP broadcasts to 47808.

So the belief that there should be an easy for "dumb" end-hosts (that may be constantly joining, leaving, moving from wired to wireless, etc.) to share arbitray security info and P2P architectures such as those provided by JXTA seem to be a logical means of doing so. Whether something like this ultimately proves to be deployable or even useful, we shall see, but there is a lot more development ahead before we can say.

(BTW: the diagram came from Chapter 4 of the JXSE 2.5 Programmers guide)

First Glorious Screencast!

2007-09-24T20:40:00.001-07:00

I could be wrong, but this is probably the only JRUBY + JXTA screencast out there. Perhaps the only JXTA screencast?

Using Wink, a free screen capture tool for Linux or Windows, I threw together a quick demo of where I'm headed in terms of of developing a JRuby abstraction layer of insulating developers from the gory details of JXTA. Nothing fancy, I basically set some environment variables, fire up jirb and create a new JXTA peergroup in about two lines of ruby.

And the best thing is you don't get to hear my grumpy, allergy-laden voice!

Setting up a PeerTAB Dev Environment

2007-09-23T17:20:00.000-07:00

I created wiki page details the following steps necessary to set up your own peertab development environment, including:

OS Installation (I used Debian Etch but it doesn't matter) and additional packages
JRE/JDK (again I'm using the Java 5 packages
Retrieving the source tree via subversion
Installing JRuby
Installing JXTA
Running a simple hello world script I wrote several months ago
Configuring fprobe and flow-tools (NetFlow will be the first EventSource) to be added to the network

Why JXTA? Updated JXTA Developer Documentation!

2007-09-22T14:30:00.000-07:00

Back in May I started looking at what Open Source P2P APIs were available. P2PS looked promising (and even still under active development) since JXTA is probably overkill for this project, but there simply wasn't a critical mass of developers/developmentr compared to what was available for JXTA. As a result there weren't enough sample apps to look or solid programmer documentation. Free Pastry was also a possibility and had more sample apps, but didn't have a rich set of security functionality. Of course if the toolset is designed correctly, it should be easy to swap in different P2P APIs (for example, if the JXTA Ruby bindings become mature enough if we want to deploy embedded nodes that don't require Java)

I was pleasantly surprised when I visited JXSE Java.net Page today (JXSE is simply the Java SE/EE implementation, as opposed to CLDC/J2ME) I was pleased to see that a new version of the programmers guide was only released a couple of weeks ago. This is good news. JXSE is currently at verison 2.5_rc3 so given the early state we are at it makes sense to go with this.

Concept Document Available

2007-09-22T07:28:00.000-07:00

Although nothing has actually been written on this document since June, I finally put up an overview/concept document I wrote (with the help of a few others, Gadi Evron in particular) to help define the problem space.

...it does not take much creativity to imagine a toolset that utilizes the types of networks used by Skype, Gnutella, or Bittorrent to provide a global view of Internet threat activity where users might choose to view, contribute, or search security information sources as easily as they would share a directory on their filesystem or download MP3’s. Initially, this network might only provide equivalent data as DShield (sources, protocols, and ports) with several obvious advantages made possible through its distributed architecture: direct (i.e. non-aggregated) access to collection nodes, the lack of a single data owner, flexible group membership (and policy definition) whereby organizations and collections of individuals to share information within an Enterprise across the Internet.

A PDF version is also available, unless you are too scared to read PDFs anymore.