[NETFRAME] Net::Packet::Dump / Net::Frame::Dump::Online - Concurrent Simultaneous Capture Limit?

[Darien Kindlund]

> Or capture the traffic via a straight tcpdump session and write it to
> a pcap file, then process it offline.

The problem is: I need this processing in close-to-realtime as possible.

> /usr/sbin/tcpdump -r $OLDSNIFF -w synonlysniffs/$NEWSNIFF
> 'tcp[tcpflags] & (tcp-syn|tcp-fin) !=0'

Yeah, I see your point, but here are the issues with this approach:

1) Making system() calls to /usr/sbin/tcpdump via perl isn't ideal.
For a variety of reasons.
2) For example, we need to programmatically start/stop tcpdump within
perl.  Why?  Because if we just left a long running tcpdump process,
then the amount of data generated by the .pcap would (eventually) fill
the disk.
3) The other thing is: let's say you do decide to create a
long-running tcpdump process to a single .pcap.  Let's say you're okay
with it generating 200MB .pcap files and then rotating thereafter.
(a) When we extract packets out of the .pcap, we specify *specific*
time ranges.  Now, as the file grows to 200MB, this extraction process
will take longer and longer to complete (due to I/O file seeking) --
until we rotate the .pcap.  (b) Then, we have to deal with issues of
time ranges that span multiple rotated .pcap files -- not too
difficult, but a pain nonetheless.

So, based upon these issues, I think it makes sense to capture only
what you need to begin with -- otherwise as the network data rates
increase (10Mb->100Mb->1Gb->10Gb), your disk I/O ends up being
enormous.

Some sort of middle ground, would be to dedicate some sort of
(sufficiently large) ring buffer in RAM (maybe as a .pcap on a
ramdisk?), and then have tcpdump write to the buffer and all the perl
processes read from the buffer.  Then the issue of figuring out how to
deal with time ranges that wrap the ring buffer could be challenging.

At the end of the day, I think I'm still left with trying out the
original Net::Frame::Dump::Online approach and take the large CPU hit.
 Initial testing tells me that Net::Frame::Dump is better than
Net::Packet::Dump, but I'll certainly post my findings to the list
once I'm finished testing.

Thanks for the suggestions though; if you have other thoughts/ideas,
I'd certainly be interested.

-- Darien