Page 1 of 1

File Parsing Back On, Database Re-arch

Posted: Mon May 03, 2004 4:40 pm
by bobzilla
File postings will not be parsed automatically again until some time this evening. We're still in the midst of a rather extensive data model re-arch to deal with scalability. You don't need to re-upload, I'll run them once parsing is ready to go again.

Sorry for the pause in service, we thank you for your patience.

(edit)
We're back to realtime parsing. I'm going through the backlog of files.

Re: File Parsing Back On, Database Re-arch

Posted: Thu May 06, 2004 1:31 am
by ccie4526
(edit)
We're back to realtime parsing. I'm going through the backlog of files.
Hmm, you sure?

My job 20040504-00077 appears to have hung up and died in parsing.
Just for grins, uploaded again today as 20040505-00049 and it's allegedly triangulating now. Hmmm.

...Ron

Posted: Thu May 06, 2004 5:32 am
by bobzilla
Yup, I'm still tuning things with the new table structures. Your file's gone through now. There's still some things i need to do to make it faster, but we're getting there.

Uploads not showing in maps

Posted: Mon May 10, 2004 8:25 pm
by ryckz-forum
While my recent uloads show as 100%, any posts after 2004/05/02 are not showing on the map yet. Is this normal? I am also confused by the "Total New Location" column in my uploads. It that a daily total? A copy of my uploads is available here: http://www.ryckz.com/wigleuploads.htm

Thanks for the great service!

the delay at work in your life

Posted: Tue May 11, 2004 2:08 am
by uhtu
as noted here, the webmaps are only updated periodically. Download one of the many wigle.net clients for realtime and interactive acess to the data!

fireside chat

Posted: Tue May 11, 2004 2:15 am
by bobzilla
While my recent uloads show as 100%, any posts after 2004/05/02 are not showing on the map yet. Is this normal?
Yes, the web map is backed by a large process that re-cache's data once every few days. I spose I should make that part of the nightly processing.
I am also confused by the "Total New Location" column in my uploads. It that a daily total?
A "location" is more like an "observation". So if you're file logged seeing a network at 19 slightly different gps locations with different signal strengths and such, that would be 19 different "locations".
Thanks for the great service!
Thank you for helping make it work! :)

Gotcha!

Posted: Fri May 14, 2004 3:31 pm
by ryckz-forum
Thank you for the detailed replies uhtu and bobzilla. Now that I know my uploads are actually working, I'll try to send more. Many of us noobs are probably initially inclined to think that when the parsing process is competed that it implies that the data will be represented immediately in the maps. I see now that it is a seperate process. Maybe a short note on the upload page could help you from needing to answer this question quite as often. While some of us actually do try to read as much as possible before asking questions, there is point of information overload, where we become like deer in headlights.

Thanks again! Of course more questions are forming as I write this. I'll hold back until I read more, and contemplate how to ask the questions!

note

Posted: Fri May 14, 2004 3:49 pm
by uhtu
they do show up immediately (well, when you reconnect.. there's no push.. yet ;-) in the Clients (DiGLE and JiGLE), which provide full interactive views of the dataset. you can also use DiGLE and JiGLE to view your stumble files locally, without the integration to the whole WiGLE dataset.

Posted: Thu Jun 17, 2004 12:59 am
by King_Ice_F
The file post thing seems to be stuck again. I don't know if you are experiencing massive amounts of uploads due to the wwwd, someone posted some really huge files and made a log jam, or if the automatic parsing is down again. In any case the number of queued files has only gotten larger (in case you didn't already know).

Posted: Thu Jun 17, 2004 2:21 am
by bobzilla
We're getting a lot of files, many of them very large. We get through them eventually :)

Posted: Thu Jun 17, 2004 3:47 am
by King_Ice_F
We're getting a lot of files, many of them very large. We get through them eventually :)
Thanks, I was worried that my posts were lost in cyber space. The rate we are going, we might have 2 million by the end of this WWWD.