From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <www-data@comics.unina.it>
Received: from svn.comics.unina.it (unknown [143.225.229.147])
	by huchra.bufferbloat.net (Postfix) with ESMTP id A82C1200507
	for <bismark-commits@lists.bufferbloat.net>;
	Tue, 24 May 2011 15:18:00 -0700 (PDT)
Received: from www-data by svn.comics.unina.it with local (Exim 4.69)
	(envelope-from <www-data@comics.unina.it>) id 1QP0DC-0005mI-MQ
	for bismark-commits@lists.bufferbloat.net;
	Wed, 25 May 2011 00:36:26 +0200
To: bismark-commits@lists.bufferbloat.net
From: nick@svn.comics.unina.it
Message-Id: <E1QP0DC-0005mI-MQ@svn.comics.unina.it>
Date: Wed, 25 May 2011 00:36:26 +0200
X-Mailman-Approved-At: Tue, 24 May 2011 15:22:22 -0700
Subject: [Bismark-commits] rev 347 - trunk/server/scripts
X-BeenThere: bismark-commits@lists.bufferbloat.net
X-Mailman-Version: 2.1.13
Precedence: list
List-Id: Commit log for the bismark source code
	<bismark-commits.lists.bufferbloat.net>
List-Unsubscribe: <https://lists.bufferbloat.net/options/bismark-commits>,
	<mailto:bismark-commits-request@lists.bufferbloat.net?subject=unsubscribe>
List-Archive: <https://lists.bufferbloat.net/pipermail/bismark-commits>
List-Post: <mailto:bismark-commits@lists.bufferbloat.net>
List-Help: <mailto:bismark-commits-request@lists.bufferbloat.net?subject=help>
List-Subscribe: <https://lists.bufferbloat.net/listinfo/bismark-commits>,
	<mailto:bismark-commits-request@lists.bufferbloat.net?subject=subscribe>
X-List-Received-Date: Tue, 24 May 2011 22:18:01 -0000

Author: nick
Date: 2011-05-25 00:36:26 +0200 (Wed, 25 May 2011)
New Revision: 347

Added:
   trunk/server/scripts/organize-archive.py
Modified:
   trunk/server/scripts/bdm
   trunk/server/scripts/dhcp_parser.py
Log:
file organization



Modified: trunk/server/scripts/bdm
===================================================================
--- trunk/server/scripts/bdm	2011-05-24 15:32:32 UTC (rev 346)
+++ trunk/server/scripts/bdm	2011-05-24 22:36:26 UTC (rev 347)
@@ -529,7 +529,7 @@
 		else
 			echo -ne $RED
 		fi
-		[ $(mysql -NB -u root -e "SELECT deviceid FROM DEVICES WHERE deviceid='$id'" $MYSQL_DB)  ] || printf "#"
+		# [ $(mysql -NB -u root -e "SELECT deviceid FROM DEVICES WHERE deviceid='$id'" $MYSQL_DB)  ] || printf "#"
 		printf "%s\t%s\t%s\t%s\t%s\n" $id $ip $user $ver $(date -d @$ts +"%Y-%m-%d %H:%M:%S")
 	done
 

Modified: trunk/server/scripts/dhcp_parser.py
===================================================================
--- trunk/server/scripts/dhcp_parser.py	2011-05-24 15:32:32 UTC (rev 346)
+++ trunk/server/scripts/dhcp_parser.py	2011-05-24 22:36:26 UTC (rev 347)
@@ -63,7 +63,6 @@
 
 if __name__ == '__main__':
   HOME = os.environ['HOME'] + '/'
-  #HOME = './'
   MEASURE_FILE_DIR = 'var/data/'
   LOG_DIR = 'var/log/'
   tables = {'measurement':'MEASUREMENTS','traceroute':'TRACEROUTES','hop':'TRACEROUTE_HOPS',\

Added: trunk/server/scripts/organize-archive.py
===================================================================
--- trunk/server/scripts/organize-archive.py	                        (rev 0)
+++ trunk/server/scripts/organize-archive.py	2011-05-24 22:36:26 UTC (rev 347)
@@ -0,0 +1,79 @@
+#!/usr/bin/env python
+
+import os
+import re
+import sys
+import time
+import datetime
+import shutil
+
+def clean_data(dir):
+
+    UKY_DIR = os.environ['HOME'] + '/var/archive/UKY-old'
+    devices = []
+    measurements = []
+    types = {'arp.gz':'arp',
+             'csv':'airodump',
+             'csv.gz':'airodump',
+             'events.gz':'events',
+             'filt.csv':'airodump',
+             'xml.gz':'active'}
+
+
+    files = os.listdir(dir)
+    for file in files:
+
+        if os.path.isdir(dir+file):
+            print 'skipping ' + file
+            continue
+
+
+        # move the Kentucky data out
+        match = re.search(r'[Uu]ky',file)
+        if match:
+            shutil.move(dir+file, UKY_DIR)
+
+        # get list of devices
+        match = re.search(r'[0-9A-Za-z]+',file)
+        if match:
+            dev_name = match.group()
+            
+            # list of devices
+            if dev_name not in devices:
+                devices.append(dev_name)
+
+        match = re.search(r'([0-9]{10})',file)
+        if match:
+            timestr = match.group(1)
+            date = datetime.date.fromtimestamp(float(timestr))
+            datedir = str(date.year) + '/' + str(date.month)
+
+        match = re.search(r'.*?[0-9]\.(.*)$',file)
+        if match:
+            extension = match.group(1)
+            typedir = types[extension]
+
+
+        # make directories and move files
+        datadir = dir+ dev_name + '/' + typedir + '/'+ datedir
+
+        if not os.path.exists(datadir):
+            os.makedirs(datadir)
+
+        #print dir+file + '->' + datadir
+        shutil.move(dir+file,datadir)
+
+
+    for dev in devices:
+        print dev
+        
+
+            
+
+
+if __name__ == '__main__':
+    HOME = os.environ['HOME'] + '/'
+    MEASURE_FILE_DIR = 'var/data/'
+    ARCHIVE_DIR = HOME + MEASURE_FILE_DIR + 'old/'
+
+    clean_data(ARCHIVE_DIR)


Property changes on: trunk/server/scripts/organize-archive.py
___________________________________________________________________
Name: svn:executable
   + *