Page 1 of 1

Convert WIGLE Query HTML into a CSV file for mapping

Posted: Wed Feb 09, 2005 4:38 pm
by stavr0
First, if you don't have the GNU AWK tool, here is where to get it for the Win32 platform: http://unxutils.sourceforge.net/

Then save your WiGLE query as a plain HTML file (In MSIE: File,Save As, Web Page HTML only)

The following script should be saved as wig2gps.awk

Code: Select all

BEGIN { # Configure AWK to treat HTML tags as record delimiter # RT always contains the last parsed record delimiter # so we can easily split up the TAG and attributes # # The only problem is that RT looks ahead at the # upcoming tag so we have to store the last parsed # tag before examining the data: # # <TAG> iter1 $0="" RT="<TAG>" # data # </TAG> iter2 $0="data" RT="</TAG>" # # RS = "<[^>]+>" FS = "\n" ORS="\n" OFS="," print "name", "desc", "latitude", "longitude", "snr", "wep" } ####################################################################### { if (tagname=="table") { # start of a table tablecount++ tablenest++ row = 0 col = 0 } if (tagname=="/table") { # end of a table tablenest-- row = 0 col = 0 } if (tagname=="tr") { # start a row row++ col=0 } if (tagname=="/tr") { # end of a row col=0 colnest=0 # print what we got if (wlat!=0) print wname,wdesc,wlat,wlon,wqos,wwep } if (tagname=="td") { # start of a cell col++ colnest++ } if (tagname=="/td") { # end of a cell colnest-- } # collect data for later printing when we encounter /TR tag # # We are interested in columns 2,3,12,13 of the # second non-nested table of this HTML syntax # if ( (tablecount==2) && (tablenest==1)&&(row>1)&&(colnest>0)) { if (col==2) wdesc = $0 # BSSID if (col==3) wname = $0 # ssid if (col==9) wdisc = $0 # discovery date # WEP if (col==11) { if ($0=="N") wwep="" else if ($0=="Open") wwep="" else wwep = $0 } if (col==12) wlat = $0 # latitude if (col==13) wlon = $0 # longitude if (col==15) wtim = $0 # last update if (col==16) wchan = $0 # channel if (col==19) wqos = $0>0?$0:"" # qos } # Capture HTML tag for next iteration # match(RT, "<([^ \r\n\t>]+)([^>]*)>",a) tagname = a[1] attcount = split(a[2],attributes," ") opentag = match(RT, "^<[^/!]") ## eat comments if ( tagname == "!--") { while (match(RT,"-->")==0) { getline } } }
Convert your query into a CSV file as follows

Code: Select all

gawk -f wig2gps.awk wiglequery.html > wiglequery.csv
The resulting .CSV is accepted by http://www.gpsvisualizer.com or Microsoft Street&Trips data import function.

(Updated Mar 2 2005, QOS and WEP parameter)