From m.c.crockett at roadrunner.com Fri Feb 1 17:52:47 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Fri, 1 Feb 2008 15:52:47 -0800 Subject: [wxqc] Weather Station Location Message-ID: I'm new to the Citizen Weather Observer Program. Looking at the quality control information for my station, CW9691; I am beginning to wonder whether I have provided the correct latitude and longitude. I have a Data Instruments Vantage Pro2 consisting of a console and an integrated sensor suite (ISS). The barometric pressure sensor is in the console and not co-located with the other weather sensors in the ISS. The location that I provided was for the ISS while the elevation was for the console containing the barometric pressure sensor. The evaluation of my barometric data appears to be based on the location of my ISS and uses old USGS topographical data for elevation. The maps date from the Fifties before the topology was changed and any houses were built. Perhaps, I'm being overly concerned because my property sits on a steep hill with a significant difference in elevation between the lowest and highest points. Before I begin mucking about with attempting to adjust the barometric pressure should I change the location reported for my station to that of the barometer? Merton Campbell Crockett m.c.crockett at roadrunner.com From kdmiller at oldsgmail.com Fri Feb 1 18:58:24 2008 From: kdmiller at oldsgmail.com (Keith Miller) Date: Fri, 01 Feb 2008 19:58:24 -0500 Subject: [wxqc] Weather Station Location In-Reply-To: Message-ID: <9D2A3A1DE0E942699DCC48E5BB0CF122@sauron> > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of Merton > Campbell Crockett > Sent: Friday, February 01, 2008 6:53 PM > To: wxqc at lists.gladstonefamily.net > Subject: [wxqc] Weather Station Location > > > I have a Data Instruments Vantage Pro2 consisting of a > console and an integrated sensor suite (ISS). The barometric > pressure sensor is in the console and not co-located with > the other weather sensors in the ISS. The location that I > provided was for the ISS while the elevation was for the > console containing the barometric pressure sensor. > > The evaluation of my barometric data appears to be based on the > location of my ISS and uses old USGS topographical data for > elevation. The maps date from the Fifties before the topology was > changed and any houses were built. > The evaluation doesn't use the NED elevation, it uses the elevation you entered when signing up. > Perhaps, I'm being overly concerned because my property sits on a > steep hill with a significant difference in elevation between the > lowest and highest points. > How much of an elevation difference is there between the console and ISS? > Before I begin mucking about with attempting to adjust the > barometric pressure should I change the location reported > for my station to that of the barometer? > Generally it's best to use the barometer elevation, in this case the console. Keith -- CW5250 => http://weather.stadhaugh.com http://weather.gladstonefamily.net/site/C5250 From k8cyj at arrl.net Fri Feb 1 21:53:59 2008 From: k8cyj at arrl.net (Rex Pirkle) Date: Fri, 1 Feb 2008 21:53:59 -0600 Subject: [wxqc] MesoWest Station "Latency Stats" In-Reply-To: Message-ID: <001d01c8654f$41b594a0$6401a8c0@DJSSJ661> Can someone explain what the "Latency Stats" mean on MesoWest Station reports, and how this influences weather data report quality? For example: http://www.met.utah.edu/cgi-bin/droman/latency.cgi?stn=AR863 I have no clue what these stats mean, or what is considered 'good' or 'bad'. Concerning another unrelated question, I read this in CWOP news: "Jan 16, 2008 - There is a new recommendation on the maximum data packet rate that CWOP members should use to send their data. This is discussed under item # 1 of the CWOP FAQ linked from the upper right. Note that if you are sending packets more often than one every 10 minutes, you should reduce your packet rate to that and you should select the minute you transmit as pointed out in item # 1 of the CWOP FAQ." It seems to me there was a once a plan proposing to increase the CWOP max data packet interval to 5 minutes. Is that plan now abandoned? Best regards, Rex Pirkle K8CYJ-5/AR863 From dshelms at comcast.net Sat Feb 2 07:12:23 2008 From: dshelms at comcast.net (Dave Helms) Date: Sat, 02 Feb 2008 08:12:23 -0500 Subject: [wxqc] MesoWest Station "Latency Stats" In-Reply-To: <001d01c8654f$41b594a0$6401a8c0@DJSSJ661> References: <001d01c8654f$41b594a0$6401a8c0@DJSSJ661> Message-ID: <47A46C37.6070103@comcast.net> Hi Rex, The latency is the valid time of the data, as encoded by FINDU before getting picked up by MADIS, and the receive time at Mesowest. Data are delivered to Mesowest from MADIS every 15 minutes, so if you have a 10 minute update interval the latency is alternatively a little more than 5 and 10 minutes. In your case, you are delivering every 5 minutes to APRS-IS/FINDU, and the effective latency is about 5-8 minutes to Mesowest/University of Utah. Here is the short story: There was some discussion on going to a 1 minute update interval between Gerry and I, but that was just talk. The policy has been to limit the update interval to 5 minute updates, but CWOP/APRSWXNET's success (growth) has strained the servers and it is apparent that we need to throttle back to 10 minute update limits until we can better manage the "CW" data load on the APRS-IS. We are also trying to manage when the data are uploaded, for instance, for CW0351, I configured WeatherDisplay upload at H+01 minutes past the hour, and H+11, H+21, etc. Likewise for my second station, CW0350, where I configured WeatherLink to upload at H+00, H+10, etc. Interesting enough, my latency on at Mesowest now falls into two bins, about 7-8 minutes, and about 12-13 minutes of data latency; this an artifact of the 15 minute data exchange between MADIS and Mesowest (half the time my 10 minute data is delivered just in time at MADIS before the exchange with Mesowest resulting in the lower wait time/latency, and half the time my data get to MADIS "early" and waits about for the next update within MADIS resulting in the longer latency), see here: http://www.met.utah.edu/cgi-bin/droman/latency.cgi?stn=c0351 Think of how your time to work or when traveling varies as a function of switching between buses/trains/planes; as your time interval increases waiting for the next bus/train/plane, this increases your time getting to your sestination. Observation latency is important issue for many applications which have short data assimilation cut-off windows. Mesowest monitors the "Obs within 12 minutes of the hour" I assume because they generate an analysis field at the top of the hour and they want "fresh data" that have valid time within this 12 minute window to diminish "time smear" of the data within the analysis. The 12 minute before the hour time window is traditionally when the ASOS/AWOS data are received. All analysis and numerical weather prediction applications of observations at NCEP, in the NWS Weather Forecast Offices and elsewhere have time windows where data are accepted. If the observation's latency is excessive, the data are excluded from the application (they missed the bus!). Hope this helps, Dave CW0351 (and CW0350) Rex Pirkle wrote: >Can someone explain what the "Latency Stats" mean on MesoWest Station >reports, and how this influences weather data report quality? >For example: >http://www.met.utah.edu/cgi-bin/droman/latency.cgi?stn=AR863 >I have no clue what these stats mean, or what is considered 'good' or >'bad'. > >Concerning another unrelated question, I read this in CWOP news: >"Jan 16, 2008 - There is a new recommendation on the maximum data packet >rate that CWOP members should use to send their data. This is discussed >under item # 1 of the CWOP FAQ linked from the upper right. Note that if >you are sending packets more often than one every 10 minutes, you should >reduce your packet rate to that and you should select the minute you >transmit as pointed out in item # 1 of the CWOP FAQ." > >It seems to me there was a once a plan proposing to increase the CWOP >max data packet interval to 5 minutes. Is that plan now abandoned? > >Best regards, > >Rex Pirkle >K8CYJ-5/AR863 > > >_______________________________________________ >wxqc mailing list >Post messages to wxqc at lists.gladstonefamily.net >To unsubcribe or change delivery options, please go to: >http://server.gladstonefamily.net/mailman/listinfo/wxqc > >The contents of this message are the responsibility of the author. > > > > From brewmaster2 at charter.net Sat Feb 2 10:09:05 2008 From: brewmaster2 at charter.net (Jim & Karen Stuntz) Date: Sat, 2 Feb 2008 08:09:05 -0800 Subject: [wxqc] MesoWest Station "Latency Stats" In-Reply-To: <001d01c8654f$41b594a0$6401a8c0@DJSSJ661> References: <001d01c8654f$41b594a0$6401a8c0@DJSSJ661> Message-ID: <001e01c865b5$efc1ca80$cf455f80$@net> Rex, I can't help on the stats but I have noticed that the packet rate may have changed. I also have the Vantage Pro 2 and have been sending every 5 minutes for almost 2 years now, but I have noticed that data only gets posted sometimes every 10 minutes. This used to be a concern to me as I was having datalogger problems with the USB port. I switched back to serial and things have been doing great. I also check the site from my office and it still sends a cringe when I see the time over 5 minutes. I would imagine that CWOP is getting really busy now, beyond their wildest dreams, and as usual equipment is lagging behind due to costs involved. They still do a great job and service to us all. Jim -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Rex Pirkle Sent: Friday, February 01, 2008 7:54 PM To: wxqc at lists.gladstonefamily.net Subject: Re: [wxqc] MesoWest Station "Latency Stats" Can someone explain what the "Latency Stats" mean on MesoWest Station reports, and how this influences weather data report quality? For example: http://www.met.utah.edu/cgi-bin/droman/latency.cgi?stn=AR863 I have no clue what these stats mean, or what is considered 'good' or 'bad'. Concerning another unrelated question, I read this in CWOP news: "Jan 16, 2008 - There is a new recommendation on the maximum data packet rate that CWOP members should use to send their data. This is discussed under item # 1 of the CWOP FAQ linked from the upper right. Note that if you are sending packets more often than one every 10 minutes, you should reduce your packet rate to that and you should select the minute you transmit as pointed out in item # 1 of the CWOP FAQ." It seems to me there was a once a plan proposing to increase the CWOP max data packet interval to 5 minutes. Is that plan now abandoned? Best regards, Rex Pirkle K8CYJ-5/AR863 _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. From k8cyj at arrl.net Sat Feb 2 14:14:41 2008 From: k8cyj at arrl.net (Rex Pirkle) Date: Sat, 2 Feb 2008 14:14:41 -0600 Subject: [wxqc] MesoWest Station "Latency Stats" Message-ID: <001e01c865d8$3f86a870$6401a8c0@DJSSJ661> Dave and Jim, Thanks for your explanation and comments; the latency statistics are now clear to me. I have throttled back AR863 to a 10 min report, commencing at 3 minutes past the hour as recommended. Even so, it does not make sense to me to use a 10 min data feed rate. Since the MADIS tick interval is every 15 minutes, it seems to me that the station report rates should be some integer factor of 15 (i.e. 1 min, 3 min, 5 min, or 15 min) for proper synchronization. At a 10 min report interval, isn't every odd numbered station report guaranteed to be skewed by at least five minutes from the real-time data point measurement? If APRS-IS servers can't handle the load from 5 min reports, it seems to me that a 15 min interval would be the next best choice for synchronization. In my mind, a 10 min report rate is out of step with what MADIS is doing. I do not see the advantage of 10 min rate over 15 min rate. Maybe my view of this is too na?ve. In general, my station well behaved until a big gradient develops in the pressure or temp. I usually get a few error reports whenever a front rolls through or the temp changes drastically in short period of time. I'm guessing that this is mainly a symptom of the latency issue -- just a few data points lagging into the wrong time slots. Anyway, I'm just trying to get a better grasp on how the QC system works as a whole. Best regards, Rex Pirkle From dshelms at comcast.net Sat Feb 2 16:12:58 2008 From: dshelms at comcast.net (dshelms at comcast.net) Date: Sat, 02 Feb 2008 22:12:58 +0000 Subject: [wxqc] MesoWest Station "Latency Stats" Message-ID: <020220082212.17950.47A4EAEA000769CB0000461E22069984999C03040A089C0B@comcast.net> Hi Rex, We need to work with the APRS-IS folks to manage the system load, and going from 5 minute to 10 minutes gets the job done in that regard. Moving back to 15 minutes runs the risk of a maximum of a 30 minute latency between the MADIS cycle and the APRS-IS delivery of the data. I'd caution against that. Dave CW0351 -------------- Original message ---------------------- From: "Rex Pirkle" > Dave and Jim, > Thanks for your explanation and comments; the latency statistics are now > clear to me. > I have throttled back AR863 to a 10 min report, commencing at 3 minutes > past the hour as recommended. > > Even so, it does not make sense to me to use a 10 min data feed rate. > Since the MADIS tick interval is every 15 minutes, it seems to me that > the station report rates should be some integer factor of 15 (i.e. 1 > min, 3 min, 5 min, or 15 min) for proper synchronization. At a 10 min > report interval, isn't every odd numbered station report guaranteed to > be skewed by at least five minutes from the real-time data point > measurement? > > If APRS-IS servers can't handle the load from 5 min reports, it seems to > me that a 15 min interval would be the next best choice for > synchronization. In my mind, a 10 min report rate is out of step with > what MADIS is doing. I do not see the advantage of 10 min rate over 15 > min rate. Maybe my view of this is too na?ve. > > In general, my station well behaved until a big gradient develops in the > pressure or temp. I usually get a few error reports whenever a front > rolls through or the temp changes drastically in short period of time. > I'm guessing that this is mainly a symptom of the latency issue -- just > a few data points lagging into the wrong time slots. Anyway, I'm just > trying to get a better grasp on how the QC system works as a whole. > > Best regards, > Rex Pirkle > > > > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From dshelms at comcast.net Sat Feb 2 16:12:58 2008 From: dshelms at comcast.net (dshelms at comcast.net) Date: Sat, 02 Feb 2008 22:12:58 +0000 Subject: [wxqc] MesoWest Station "Latency Stats" Message-ID: <020220082212.17950.47A4EAEA000769CB0000461E22069984999C03040A089C0B@comcast.net> Hi Rex, We need to work with the APRS-IS folks to manage the system load, and going from 5 minute to 10 minutes gets the job done in that regard. Moving back to 15 minutes runs the risk of a maximum of a 30 minute latency between the MADIS cycle and the APRS-IS delivery of the data. I'd caution against that. Dave CW0351 -------------- Original message ---------------------- From: "Rex Pirkle" > Dave and Jim, > Thanks for your explanation and comments; the latency statistics are now > clear to me. > I have throttled back AR863 to a 10 min report, commencing at 3 minutes > past the hour as recommended. > > Even so, it does not make sense to me to use a 10 min data feed rate. > Since the MADIS tick interval is every 15 minutes, it seems to me that > the station report rates should be some integer factor of 15 (i.e. 1 > min, 3 min, 5 min, or 15 min) for proper synchronization. At a 10 min > report interval, isn't every odd numbered station report guaranteed to > be skewed by at least five minutes from the real-time data point > measurement? > > If APRS-IS servers can't handle the load from 5 min reports, it seems to > me that a 15 min interval would be the next best choice for > synchronization. In my mind, a 10 min report rate is out of step with > what MADIS is doing. I do not see the advantage of 10 min rate over 15 > min rate. Maybe my view of this is too na?ve. > > In general, my station well behaved until a big gradient develops in the > pressure or temp. I usually get a few error reports whenever a front > rolls through or the temp changes drastically in short period of time. > I'm guessing that this is mainly a symptom of the latency issue -- just > a few data points lagging into the wrong time slots. Anyway, I'm just > trying to get a better grasp on how the QC system works as a whole. > > Best regards, > Rex Pirkle > > > > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From m.c.crockett at roadrunner.com Sat Feb 2 17:05:17 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sat, 2 Feb 2008 15:05:17 -0800 Subject: [wxqc] Weather Station Location In-Reply-To: <9D2A3A1DE0E942699DCC48E5BB0CF122@sauron> References: <9D2A3A1DE0E942699DCC48E5BB0CF122@sauron> Message-ID: On 01 Feb 2008, at 16:58:24, Keith Miller wrote: >> -----Original Message----- >> From: wxqc-bounces at lists.gladstonefamily.net >> [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of Merton >> Campbell Crockett >> Sent: Friday, February 01, 2008 6:53 PM >> To: wxqc at lists.gladstonefamily.net >> Subject: [wxqc] Weather Station Location >> >> > >> I have a Data Instruments Vantage Pro2 consisting of a >> console and an integrated sensor suite (ISS). The barometric >> pressure sensor is in the console and not co-located with >> the other weather sensors in the ISS. The location that I >> provided was for the ISS while the elevation was for the >> console containing the barometric pressure sensor. >> >> The evaluation of my barometric data appears to be based on the >> location of my ISS and uses old USGS topographical data for >> elevation. The maps date from the Fifties before the topology was >> changed and any houses were built. >> > > The evaluation doesn't use the NED elevation, it uses the elevation > you entered when signing up. Due to my "newbie" status, I'm, perhaps, overly concerned with being accurate. I've spent a lot of time looking at maps since Christmas trying to determine the exact location of my ISS, it's elevation, and the elevation of my console. One thing that I noticed was that there was a variance in the registration of the street and parcel information to the USGS survey data. I had to make an educated guess at the elevation that would minimize the cuts and fills needed to provide a level grade for the foundation. >> Perhaps, I'm being overly concerned because my property sits on a >> steep hill with a significant difference in elevation between the >> lowest and highest points. >> > > How much of an elevation difference is there between the console > and ISS? The ISS is 1.5 meters above grade and the console is roughly 2 meters above grade. >> Before I begin mucking about with attempting to adjust the >> barometric pressure should I change the location reported >> for my station to that of the barometer? >> > Generally it's best to use the barometer elevation, in this > case the console. I used the "sea-level equivalent" barometric pressure instead of the altimeter setting at the Pt Mugu, Naval Air Warfare Center. I've re- adjusted my barometer based on the latter and MADIS is much happier with the results: two thumbs up. Merton Campbell Crockett > > Keith > -- > CW5250 => http://weather.stadhaugh.com > http://weather.gladstonefamily.net/site/C5250 > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. Merton Campbell Crockett m.c.crockett at roadrunner.com From m.c.crockett at roadrunner.com Sat Feb 2 18:02:37 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sat, 2 Feb 2008 16:02:37 -0800 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> Message-ID: <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> On 31 Jan 2008, at 03:44:32, Steve Dimse wrote: > > On Jan 31, 2008, at 12:47 AM, Merton Campbell Crockett wrote: > >> Assume that a given ISP provided support to only 4 CWOP Clients. >> Assume that all of the CWOP Clients had stable clocks with no drift >> and reported at the same interval. In this scenario, it is highly >> probable that although each was querying for rotate.aprs.net each >> CWOP >> Client would connect to the same CWOP Server each reporting period, >> i.e. the load is distributed from the perspective of the servers but >> not from the perspective of the client. >> >> Depending upon the distribution CWOP Clients around the world, it is >> reasonable to assume that there will be periodic skewing of activity >> to a subset of the CWOP Servers. >> > CWOP is a minority of the use of the APRS IS in general and > rotate.aprs.net in particular. APRS is a ham radio system that > provides real time tracking, many of those people use rotate for their > connections. These are stable bidirectional connections, a user > connects and stays connected for very long periods of time because > they constantly receive data through the connection, they do not just > use it to send data. Their coming and going (and therefore their DNS > requests) does not follow any temporal pattern. > > Even without this randomization, CWOP users are not all sending data > constantly, at any time someone could turn on or off their computer, > drop or regains their internet connection, etc. With 4000+ users in a > given day, 3000+ active in any five minute period, the odds of a > stable DNS sequence are remote. > > It is not surprising someone has five packets through a single server > in a row, just as you can get five heads in a row when you flip a > coin. Not probable, but over time long runs are inevitable in any > random event. However, when you look at 24 hours of data from someone > using rotate and see every packet goes through a single server, the > odds of this being random become astronomical. Something is wrong, > either the OS does not obey the cache timeout, or the program simply > does the name lookup once and reuses the same IP on each connect. My interest in this thread was piqued by observing at the daily event log for CW9691. I had seen the note about using rotate.aprs.net and that it was configured with 4 A records. As I have a BIND name server running on my system and have only one application, wvcwopd, that queries for the name server, I would have expected to see each entry going to the next server in the list. The reason for the same server being used repeatedly became clear when I noticed the TTL on rotate.aprs.net was set to 60 seconds and that the TTL on each of the NS records was 316 seconds. So each time wvcwopd needed to communicate, a new rotate.aprs.net list had to be obtained. Without digging out my statistics text book, roughly 60 percent of the time the CWOP Server would be the same as the last one used. From the perspective of the CWOP Client it doesn't matter if it constantly uses the same server. It's more important to ensure that the load is balanced between the CWOP Servers. Most of my experience in using DNS to load-level activity has involved mail relays. I found that a large TTL worked better for keeping the load balanced but that may be the result of mail relays being better behaved and establishing longer connections. Using DNS to load-level Squid Proxy activity wasn't quite as successful after Microsoft introduced Windows 2000 and Windows XP. To "improve the user's experience", they added a feature that "remembered" how it last accessed a given URL. Another "feature" was "negative caching". If you couldn't establish a connection to a system that information would be cached for approximately 6 hours. During that time the system would be unreachable. Merton Campbell Crockett m.c.crockett at roadrunner.com From k8cyj at arrl.net Sat Feb 2 19:18:25 2008 From: k8cyj at arrl.net (Rex Pirkle) Date: Sat, 2 Feb 2008 19:18:25 -0600 Subject: [wxqc] MesoWest Station "Latency Stats" Message-ID: <001f01c86602$b02ac5a0$6401a8c0@DJSSJ661> Hi Dave, After adjusting to a 10 min report, I see that the MesoWest "Tabular Listing" automatically adjusted to a 20-minute log interval rather than 15 minutes: http://www.met.utah.edu/cgi-bin/droman/meso_base.cgi?stn=ar863 In summary, with a five minute upload, they were picking the best 'one out of three' data points in a 15 minute span, and tossing two data points out. With the 10-minute interval, they accept the better of two data points in 20 minutes, and dismiss just one. So, if two old stations move from a 5 to 10 min report rate, it make enough room in the APRS-IS pipeline for one more new station (with no change in APRS-IS server resources). That makes perfect sense to me. I understand your comment about a risk of 30 minute latency, and the intent of the 10 min report rate is now clear to me -- it's the best trade off to maximize the MADIS cycle while balancing the APRS-IS load. Best regards, Rex Pirkle K8CYJ-5/AR863 Hi Rex, We need to work with the APRS-IS folks to manage the system load, and going from 5 minute to 10 minutes gets the job done in that regard. Moving back to 15 minutes runs the risk of a maximum of a 30 minute latency between the MADIS cycle and the APRS-IS delivery of the data. I'd caution against that. Dave CW0351 From mark at markwyman.com Sat Feb 2 19:49:00 2008 From: mark at markwyman.com (Mark Wyman) Date: Sat, 2 Feb 2008 20:49:00 -0500 Subject: [wxqc] Unbuntu, replacement for WeatherLink Message-ID: <00f201c86606$f3222fc0$01fea8c0@MWYMANS> Hi All, Started making the switch to Unbuntu (50/50 with Windows) and I would like to continue to upload data while in a Ubuntu session. What (hopefully) free software is recommended for this purpose? If not free, what other tools are available? Thanks. -Mark No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.18/1255 - Release Date: 2/1/2008 9:59 AM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080202/558695da/attachment.html From garbal54 at gmail.com Sat Feb 2 19:54:06 2008 From: garbal54 at gmail.com (Gary B) Date: Sat, 2 Feb 2008 21:54:06 -0400 Subject: [wxqc] Unbuntu, replacement for WeatherLink References: <00f201c86606$f3222fc0$01fea8c0@MWYMANS> Message-ID: <002b01c86607$aa0b9870$d951de18@home4thow111j8> Weather Display isn't free ($79.00) but there is a version for Linux and I can't say enough about what great software this is. ----- Original Message ----- From: Mark Wyman To: 'Discussion of weather data quality issues' Sent: Saturday, February 02, 2008 9:49 PM Subject: [wxqc] Unbuntu, replacement for WeatherLink Hi All, Started making the switch to Unbuntu (50/50 with Windows) and I would like to continue to upload data while in a Ubuntu session. What (hopefully) free software is recommended for this purpose? If not free, what other tools are available? Thanks. -Mark No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.18/1255 - Release Date: 2/1/2008 9:59 AM ------------------------------------------------------------------------------ _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080202/38900c36/attachment.html From m.c.crockett at roadrunner.com Sat Feb 2 19:59:59 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sat, 2 Feb 2008 17:59:59 -0800 Subject: [wxqc] CWOP Server: Reporting Constraints Message-ID: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> Does a CWOP Server impose a constraint that only one report can be transmitted per connection? I have a Davis Instruments Vantage Pro2. The software provided by Davis Instruments with the WeatherLink Data Logger does not support sending data to a CWOP Server when the host is running Mac OS X. To support reporting observations to CWOP from my system; I compiled, linked, and installed wview 3.4.1. I configured wview to obtain updates from the WeatherLink Data Logger every 5 minutes. I configured it to send CWOP reports every 15 minutes. This doesn't work. I observe the wvcwopd daemon connecting to a CWOP Server every 5 minutes after it probes the WeatherLink Data Logger for the latest update. Before I start looking at the source code, I would like to know if the CWOP Servers limit the CWOP Client to one report per connection. Obviously, sending multiple reports per connection would eliminate the need to manipulate the data to be correct for the 15 minute reporting period. Merton Campbell Crockett m.c.crockett at roadrunner.com From m.c.crockett at roadrunner.com Sat Feb 2 20:18:22 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sat, 2 Feb 2008 18:18:22 -0800 Subject: [wxqc] Unbuntu, replacement for WeatherLink In-Reply-To: <002b01c86607$aa0b9870$d951de18@home4thow111j8> References: <00f201c86606$f3222fc0$01fea8c0@MWYMANS> <002b01c86607$aa0b9870$d951de18@home4thow111j8> Message-ID: <8B115C9C-72DD-4655-BE97-4A29755640B0@roadrunner.com> On 02 Feb 2008, at 17:54:06, Gary B wrote: > Weather Display isn't free ($79.00) but there is a version for Linux > and I can't say enough about what great software this is. Another possibility is wview. It is released in source code under the Linux GPL. I might have selected Weather Display but it doesn't support PowerPC- based Apple systems such as my PowerMac G5. I couldn't justify spending $2800.00+tax to replace my existing system to run a $79.00 software package. I found wview 3.4.1 that runs under Linux or Unix (Mac OS X is a BSD- based Unix). The only catch is that you have to download all the required components in addition to wview. I presume that Ubuntu provides all the software development tools that would be needed to make this a viable option. Merton Campbell Crockett > ----- Original Message ----- > From: Mark Wyman > To: 'Discussion of weather data quality issues' > Sent: Saturday, February 02, 2008 9:49 PM > Subject: [wxqc] Unbuntu, replacement for WeatherLink > > Hi All, > Started making the switch to Unbuntu (50/50 with Windows) and I > would like to continue to upload data while in a Ubuntu session. > What (hopefully) free software is recommended for this purpose? If > not free, what other tools are available? > > Thanks. > > -Mark > > No virus found in this outgoing message. > Checked by AVG Free Edition. > Version: 7.5.516 / Virus Database: 269.19.18/1255 - Release Date: > 2/1/2008 9:59 AM > > > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. Merton Campbell Crockett m.c.crockett at roadrunner.com -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080202/de00e5b5/attachment.html From kdmiller at oldsgmail.com Sat Feb 2 20:23:19 2008 From: kdmiller at oldsgmail.com (Keith Miller) Date: Sat, 02 Feb 2008 21:23:19 -0500 Subject: [wxqc] Weather Station Location In-Reply-To: Message-ID: <8E1C0561F8EE4E31B5C6C42C6DCFAEC1@sauron> > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of Merton > Campbell Crockett > Sent: Saturday, February 02, 2008 6:05 PM > To: Discussion of weather data quality issues > Subject: Re: [wxqc] Weather Station Location > > > > Due to my "newbie" status, I'm, perhaps, overly concerned with > being accurate. I've spent a lot of time looking at maps since > Christmas trying to determine the exact location of my ISS, > it's elevation, and the elevation of my console. > Nothing wrong with trying to be accurate. I think I see where your real concern was though, trying to figure out your actual elevation from out dated USGS maps. That could be difficult. > > How much of an elevation difference is there between the console > > and ISS? > > > The ISS is 1.5 meters above grade and the console is roughly > 2 meters above grade. > Not enough to make a signifigant difference. > I used the "sea-level equivalent" barometric pressure instead > of the altimeter setting at the Pt Mugu, Naval Air Warfare > Center. I've re-adjusted my barometer based on the latter > and MADIS is much happier with the results: two thumbs up. > Ah, that would make a difference. Did you adjust to what's being displayed on the console, or what is being sent to CWOP? The console displays sea-level reduced pressure... Keith -- CW5250 => http://weather.stadhaugh.com http://weather.gladstonefamily.net/site/C5250 From stsander at sblan.net Sat Feb 2 20:51:39 2008 From: stsander at sblan.net (Stan Sander) Date: Sat, 02 Feb 2008 19:51:39 -0700 Subject: [wxqc] Unbuntu, replacement for WeatherLink In-Reply-To: <00f201c86606$f3222fc0$01fea8c0@MWYMANS> References: <00f201c86606$f3222fc0$01fea8c0@MWYMANS> Message-ID: <47A52C3B.2060800@sblan.net> Mark Wyman wrote: > Hi All, > > Started making the switch to Unbuntu (50/50 with Windows) and I would > like to continue to upload data while in a Ubuntu session. What > (hopefully) free software is recommended for this purpose? If not free, > what other tools are available? > > > > Thanks. > > > > -Mark > Try having a look at http://home.comcast.net/~dshelms/cwop.html and take your pick. There are several software packages listed there. Stan N5KJT From dave at aprsfl.net Sat Feb 2 22:24:10 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sat, 2 Feb 2008 23:24:10 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> Message-ID: <01e801c8661c$9fb3d760$df1b8620$@net> Merton, There is no limit per connection, you could drop off 10 different callsigns-SSID/cwop ID's worth of weather at once per connect if you wanted.... or you even could send ten packets of the same station with the same callsign-ssid/CWOP-ID (would not do much good, since there is a 30 second duplicate timer that would trash 9 of them, but it would -still- accept them). Now you -cannot- connect to the server more than once per unique callsign-SSID/CWOP id though. Also make sure your program if you tear into the source code goes off to another server in the pool if you get a "Port Full" message along with a closed connection, if you do as some do (connect, wait x seconds, send string), that packet would never make it. Just one more sanity check we're trying to get CWOP software developers to implement. The APRS specification is available online here: http://www.tapr.org/aprs_working_group.html Here is the website that has links to and information about the APRS-IS network: http://www.aprs-is.net/overview.htm Also, the differences here in my descriptions of the ID's are this: CWOP ID's are in the format of say AP808 (my home station) Callsign-SSID's for Amateurs are such as KG4YZY-10. (my two-way repeater site) It's the amateur's callsign, with a optional number from 1-15 so we can run multiple assets on air/net with a single callsign. Seeya, Dave KG4YZY -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Merton Campbell Crockett Sent: Saturday, February 02, 2008 9:00 PM To: Discussion of weather data quality issues Subject: [wxqc] CWOP Server: Reporting Constraints Does a CWOP Server impose a constraint that only one report can be transmitted per connection? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From seth at ieee.org Sun Feb 3 04:02:44 2008 From: seth at ieee.org (Seth Cohen) Date: Sun, 03 Feb 2008 05:02:44 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <01e801c8661c$9fb3d760$df1b8620$@net> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> Message-ID: <003b01c8664b$ed588180$c8098480$@org> As somebody who wants to be a good CWOP citizen and software developer, is there some place that documents what a typical conversation with the server looks like and what messages it can return? I've looked at the APRS docs and they discuss the report format, but not the possible returns we can receive back from the server and what they mean. A good example would be the "Port Full" message mentioned in the email below. Seth KB3HHA -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Dave Anderson KG4YZY Sent: Saturday, February 02, 2008 11:24 PM To: 'Discussion of weather data quality issues' Subject: Re: [wxqc] CWOP Server: Reporting Constraints Merton, There is no limit per connection, you could drop off 10 different callsigns-SSID/cwop ID's worth of weather at once per connect if you wanted.... or you even could send ten packets of the same station with the same callsign-ssid/CWOP-ID (would not do much good, since there is a 30 second duplicate timer that would trash 9 of them, but it would -still- accept them). Now you -cannot- connect to the server more than once per unique callsign-SSID/CWOP id though. Also make sure your program if you tear into the source code goes off to another server in the pool if you get a "Port Full" message along with a closed connection, if you do as some do (connect, wait x seconds, send string), that packet would never make it. Just one more sanity check we're trying to get CWOP software developers to implement. The APRS specification is available online here: http://www.tapr.org/aprs_working_group.html Here is the website that has links to and information about the APRS-IS network: http://www.aprs-is.net/overview.htm Also, the differences here in my descriptions of the ID's are this: CWOP ID's are in the format of say AP808 (my home station) Callsign-SSID's for Amateurs are such as KG4YZY-10. (my two-way repeater site) It's the amateur's callsign, with a optional number from 1-15 so we can run multiple assets on air/net with a single callsign. Seeya, Dave KG4YZY -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Merton Campbell Crockett Sent: Saturday, February 02, 2008 9:00 PM To: Discussion of weather data quality issues Subject: [wxqc] CWOP Server: Reporting Constraints Does a CWOP Server impose a constraint that only one report can be transmitted per connection? -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5104 bytes Desc: not available Url : http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080203/cf646f90/attachment.bin From steve at dimse.com Sun Feb 3 07:10:34 2008 From: steve at dimse.com (Steve Dimse) Date: Sun, 3 Feb 2008 08:10:34 -0500 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> Message-ID: <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> On Feb 2, 2008, at 7:02 PM, Merton Campbell Crockett wrote: > I had seen the note about using rotate.aprs.net and that it was > configured with 4 A records. As I have a BIND name server running on > my system and have only one application, wvcwopd, that queries for the > name server, I would have expected to see each entry going to the next > server in the list. If I understand you correctly, you think your your bind would cache the name and then rotate through it? That may be right if the queries happen before the expiration time, say every 5 seconds. When a query occurs after the expiration time (60 seconds for rotate.aprs.net), your bind hits the aprs.net bind and relays through the order aprs.net returns. > The reason for the same server being used repeatedly became clear when > I noticed the TTL on rotate.aprs.net was set to 60 seconds and that > the TTL on each of the NS records was 316 seconds. So each time > wvcwopd needed to communicate, a new rotate.aprs.net list had to be > obtained. Without digging out my statistics text book, roughly 60 > percent of the time the CWOP Server would be the same as the last one > used. > I guess the 316 you mention is in the additional section, the TTL of the name servers themselves. This will change depending on your own dns cache. for example if you hit rotate twice in a row a couple seconds apart, even the TTL of rotate changes, down to 58 seconds in the second hit [sdimse at flathat sdimse]$ dig rotate.aprs.net ; <<>> DiG 9.2.4 <<>> rotate.aprs.net ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57515 ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 2, ADDITIONAL: 2 ;; QUESTION SECTION: ;rotate.aprs.net. IN A ;; ANSWER SECTION: rotate.aprs.net. 60 IN A 67.18.222.130 rotate.aprs.net. 60 IN A 134.173.254.38 rotate.aprs.net. 60 IN A 165.91.140.28 rotate.aprs.net. 60 IN A 64.58.200.120 ;; AUTHORITY SECTION: aprs.net. 4273 IN NS ns3.aprs.net. aprs.net. 4273 IN NS ns7.aprs.net. ;; ADDITIONAL SECTION: ns3.aprs.net. 4273 IN A 24.123.66.139 ns7.aprs.net. 4273 IN A 64.58.207.2 ;; Query time: 66 msec ;; SERVER: 128.194.254.1#53(128.194.254.1) ;; WHEN: Sun Feb 3 12:58:51 2008 ;; MSG SIZE rcvd: 165 [sdimse at flathat sdimse]$ dig rotate.aprs.net ; <<>> DiG 9.2.4 <<>> rotate.aprs.net ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8817 ;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 2, ADDITIONAL: 2 ;; QUESTION SECTION: ;rotate.aprs.net. IN A ;; ANSWER SECTION: rotate.aprs.net. 58 IN A 64.58.200.120 rotate.aprs.net. 58 IN A 67.18.222.130 rotate.aprs.net. 58 IN A 134.173.254.38 rotate.aprs.net. 58 IN A 165.91.140.28 ;; AUTHORITY SECTION: aprs.net. 4271 IN NS ns7.aprs.net. aprs.net. 4271 IN NS ns3.aprs.net. ;; ADDITIONAL SECTION: ns3.aprs.net. 4271 IN A 24.123.66.139 ns7.aprs.net. 4271 IN A 64.58.207.2 ;; Query time: 0 msec ;; SERVER: 128.194.254.1#53(128.194.254.1) ;; WHEN: Sun Feb 3 12:58:53 2008 ;; MSG SIZE rcvd: 165 Even if the TTL on the nameservers was always 316 I'm not sure where the 60% comes from. After the 60 second timeout (meaning every a cwop report is to be sent) a new list of rotate.aprs.net is obtained, and this will be in random order (because of the large, unpredictable number of other people hitting each nameserver), so there is a 25% chance of being on the same aprs.net server. Steve K4HG From steve at dimse.com Sun Feb 3 07:35:51 2008 From: steve at dimse.com (Steve Dimse) Date: Sun, 3 Feb 2008 08:35:51 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <01e801c8661c$9fb3d760$df1b8620$@net> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> Message-ID: <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> On Feb 2, 2008, at 11:24 PM, Dave Anderson KG4YZY wrote: > Merton, > > There is no limit per connection, you could drop off 10 different > callsigns-SSID/cwop ID's worth of weather at once per connect if you > wanted.... Actually this is not correct. When you connect, you give a "callsign" in this case a CWOP number, and an incorrect password of -1. This is what we call an unvalidated connection, meaning you are not an amateur radio operator. This type of connection can only send data from the callsign you logged on with. If you log on as CW9999, you can only send data from CW9999. That is all a CW station ever should do anyway, so this is no constraint. Steve K4HG From dave at aprsfl.net Sun Feb 3 08:39:55 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 09:39:55 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> Message-ID: <029f01c86672$a46eaa40$ed4bfec0$@net> >what we call an unvalidated connection, meaning you are not an amateur Yikes!!! I forgot about that. I need to quit answering e-mails when I'm tired . I'm so used to Hams asking that question when working with igates and other objects that it slipped my mind with unvalidated (cwop) connects. Seeya, Dave -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From dave at aprsfl.net Sun Feb 3 09:24:16 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 10:24:16 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <003b01c8664b$ed588180$c8098480$@org> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <003b01c8664b$ed588180$c8098480$@org> Message-ID: <02b601c86678$d7035e50$850a1af0$@net> Seth, A lot of the details are spread out across the http://www.aprs-is.net website, not really a single "comprehensive" page. Since most of the users on APRS-IS are hams, and all developers are hams that also use the network, there's a bit of understanding that goes hand in hand. There are plenty of discussions about issues on forums such as the TAPR APRS SIG. Unfortunately, CWOP developers do not use the network in this capacity, so most of the resources available for developers are not clearly or easily obtainable quickly, nor in some cases not being a amateur operator would even be given access to some of the forums. The sysops of the servers, along with the developer of the server code and the administrators of CWOP itself last year started putting a white paper together. This document addressed every issue a CWOP author would need to consider, without having to troll the net for the answers. Unfortunately, as I was the document's "ringleader" in development, I gave up on trying to complete it due to some very strong differences in opinions on a few of the policies outlined in the document. Issues such as the exact polling interval and URL to use were really big sticking points. The sysop's didn't want folks to use rotate.aprs.net (as hams use it), we wanted CWOP users to use a new rotate URL that would have been created called cwop.aprs.net that would do the same thing. This would have given us the option to move CWOP users to a separate non-ham set of servers at a later date without users having to change anything (something that truly with added growth in the past year really needs to be done). Given that the shake up with the other server group dropping support occurred just a few months before this document's development, CWOP management didn't want to make too many changes at once (even though they critically needed addressed and still do). So the document fell to the wayside out of frustration. I'd say 99% of the issues CWOP users have with the network is due to client software not following the configurations of the servers correctly. With there being no public document that assists developers aside from a few of the bullet points that have been posted in this forum, you can see the conundrum here. That's why I tried to put this document together in the first place. I'll send you a copy of the last draft of the document from last year in e-mail. The procedure for how to connect, what to expect, etc is dead on accurate in it. I'm not trying to flame anyone here, especially not the CWOP management. I should have persevered on and fought to complete the document last year versus throwing in the towel out of disgust, so I'm just as guilty for why the document isn't completed. Seeya, Dave KG4YZY Sysop third & fourth -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Seth Cohen Sent: Sunday, February 03, 2008 5:03 AM To: 'Discussion of weather data quality issues' Subject: Re: [wxqc] CWOP Server: Reporting Constraints As somebody who wants to be a good CWOP citizen and software developer, is -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From m.c.crockett at roadrunner.com Sun Feb 3 10:18:04 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 08:18:04 -0800 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> Message-ID: <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> On 03 Feb 2008, at 05:35:51, Steve Dimse wrote: > > On Feb 2, 2008, at 11:24 PM, Dave Anderson KG4YZY wrote: > >> Merton, >> >> There is no limit per connection, you could drop off 10 different >> callsigns-SSID/cwop ID's worth of weather at once per connect if you >> wanted.... > > Actually this is not correct. When you connect, you give a "callsign" > in this case a CWOP number, and an incorrect password of -1. This is > what we call an unvalidated connection, meaning you are not an amateur > radio operator. This type of connection can only send data from the > callsign you logged on with. If you log on as CW9999, you can only > send data from CW9999. That is all a CW station ever should do anyway, > so this is no constraint. Steve: Thanks for the explanation of why the password was set to -1. I am new to both weather reporting and to CWOP and may have not asked the question correctly. From what I've observed, a CWOP weather report consists of a single line of ASCII text. In this report is a time-stamp indicating the time at which the data was collected. Let's assume that I have a weather station that tabulates reports on a 5 minute interval. To minimize load on the aggregation/ingest system, it is desirable to limit connections to once every 15 minutes. Assuming that CW9999 connects at 16 minutes after the hour, can it send the 3 reports for the periods ending at 05, 10, and 15 minutes after the hour? The catch is that the 10 and 15 minute reports cannot be discarded as was suggested by one of the respondents to my initial question. If the aggregation/ingest system discards the additional reports, CW9999 must perform the aggregation of cumulative data. Does this clarify what I wanted to know or, merely, provide additional obfuscation? Merton Campbell Crockett m.c.crockett at roadrunner.com From tcrissman at cableone.net Sun Feb 3 10:20:36 2008 From: tcrissman at cableone.net (Terry Crissman) Date: Sun, 3 Feb 2008 10:20:36 -0600 Subject: [wxqc] AS554 Shutting Down for 60 to 90 days Message-ID: <000001c86680$b5512460$8f843d43@e6600> I'll be shutting my station down for 60 to 90 days due to a move. thanks Terry Crissman AC5AN (AS554) No virus found in this outgoing message. Checked by AVG Free Edition. Version: 7.5.516 / Virus Database: 269.19.19/1256 - Release Date: 2/2/2008 1:50 PM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080203/1072dbc6/attachment.html From dave at aprsfl.net Sun Feb 3 10:31:21 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 11:31:21 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> Message-ID: <02ef01c86682$35bf24c0$a13d6e40$@net> Ah, I see what you were referring to now. APRS-IS is a real time tactical network, so from the perspective of typical operation it is not desired to do this. It still creates undesired packet-per-second load on the network. Your data submissions not only goes to Findu, but up to 3500 other ham's connected to the APRS-IS get it in real time as well, so you can see how a single packet explodes into a tremendous amount of bandwidth with 16,000 stations on APRS-IS. With the case of getting your data, Steve is only going to send the last packet received at the point he creates the data packet that he sends to MADIS, so the other two reports would essentially be not used. Even Findu itself would only show the last one heard (I'm not sure if Steve uses the timestamp, most APRS client software just displays the last heard packet, he can chime in here). Seeya, Dave ----- Assuming that CW9999 connects at 16 minutes after the hour, can it send the 3 reports for the periods ending at 05, 10, and 15 minutes after the hour? The catch is that the 10 and 15 minute reports cannot be discarded as was suggested by one of the respondents to my initial question. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From steve at dimse.com Sun Feb 3 10:31:55 2008 From: steve at dimse.com (Steve Dimse) Date: Sun, 3 Feb 2008 11:31:55 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> Message-ID: <0D422FD4-C1AD-4773-96BB-FD7B87B5C592@dimse.com> On Feb 3, 2008, at 11:18 AM, Merton Campbell Crockett wrote: > Assuming that CW9999 connects at 16 minutes after the hour, can it > send the 3 reports for the periods ending at 05, 10, and 15 minutes > after the hour? The catch is that the 10 and 15 minute reports cannot > be discarded as was suggested by one of the respondents to my initial > question. > No. The APRS Internet System is a realtime system. There is no provision for sending old data, timestamps in the packet are ignored. In the example above, one minute isn't going to make a difference, but if you have access to the data at 16 minutes after the hour that is what should be sent, not the 15 minute data. Steve From Karl.Uppiano at verizon.net Sun Feb 3 11:57:09 2008 From: Karl.Uppiano at verizon.net (Karl Uppiano) Date: Sun, 03 Feb 2008 09:57:09 -0800 Subject: [wxqc] CWOP Server: Reporting Constraints References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <003b01c8664b$ed588180$c8098480$@org> <02b601c86678$d7035e50$850a1af0$@net> Message-ID: <61866A6C36E249A2A260E5B007FFF219@Inspiron> Dave, I would be very interested in having a comprehensive and authoritative specification for APRS-CWOP clients as well. I am the developer of WxService (http://mysite.verizon.net/Karl_Uppiano/wxservice.html). I have changed my implementation several times over the past year or two in response to reading updated CWOP web pages, and/or reinterpreting existing pages. Currently, my software is capable of rotating through a server list. This is the default list: rotate.aprs.net:14580;first.aprs.net:14580;second.aprs.net:14580;third.aprs.net:14580; If the first connection fails, it tries the next, then the next, and so on until a connection is successful, or the list is exhausted, in which case, the post fails. This was my understanding of the protocol the last time I saw it discussed. Another question: Which fields absolutely, positively must contain valid data? It is my understanding that wind direction, wind speed, wind gust and temperature may not be omitted, but may contain "..." if there is no data available. However, my thermometer was offline briefly, and I found CWOP recorded zero for that post instead of some kind of null value. I have modified my implementation to not post anything if temperature is not available (even if other data is available), but I need to know if this is the appropriate behavior. Regards Karl Uppiano -------------------------------------------------- From: "Dave Anderson KG4YZY" Sent: Sunday, February 03, 2008 7:24 AM To: "'Discussion of weather data quality issues'" Subject: Re: [wxqc] CWOP Server: Reporting Constraints > Seth, > > A lot of the details are spread out across the http://www.aprs-is.net > website, not really a single "comprehensive" page. Since most of the > users > on APRS-IS are hams, and all developers are hams that also use the > network, > there's a bit of understanding that goes hand in hand. There are plenty > of > discussions about issues on forums such as the TAPR APRS SIG. > Unfortunately, CWOP developers do not use the network in this capacity, so > most of the resources available for developers are not clearly or easily > obtainable quickly, nor in some cases not being a amateur operator would > even be given access to some of the forums. > > The sysops of the servers, along with the developer of the server code and > the administrators of CWOP itself last year started putting a white paper > together. This document addressed every issue a CWOP author would need to > consider, without having to troll the net for the answers. > Unfortunately, > as I was the document's "ringleader" in development, I gave up on trying > to > complete it due to some very strong differences in opinions on a few of > the > policies outlined in the document. Issues such as the exact polling > interval and URL to use were really big sticking points. The sysop's > didn't > want folks to use rotate.aprs.net (as hams use it), we wanted CWOP users > to > use a new rotate URL that would have been created called cwop.aprs.net > that > would do the same thing. This would have given us the option to move CWOP > users to a separate non-ham set of servers at a later date without users > having to change anything (something that truly with added growth in the > past year really needs to be done). Given that the shake up with the > other > server group dropping support occurred just a few months before this > document's development, CWOP management didn't want to make too many > changes > at once (even though they critically needed addressed and still do). So > the > document fell to the wayside out of frustration. > > I'd say 99% of the issues CWOP users have with the network is due to > client > software not following the configurations of the servers correctly. With > there being no public document that assists developers aside from a few of > the bullet points that have been posted in this forum, you can see the > conundrum here. That's why I tried to put this document together in the > first place. > > I'll send you a copy of the last draft of the document from last year in > e-mail. The procedure for how to connect, what to expect, etc is dead on > accurate in it. > > I'm not trying to flame anyone here, especially not the CWOP management. > I > should have persevered on and fought to complete the document last year > versus throwing in the towel out of disgust, so I'm just as guilty for why > the document isn't completed. > > Seeya, > Dave > KG4YZY > Sysop third & fourth > > > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Seth Cohen > Sent: Sunday, February 03, 2008 5:03 AM > To: 'Discussion of weather data quality issues' > Subject: Re: [wxqc] CWOP Server: Reporting Constraints > > As somebody who wants to be a good CWOP citizen and software developer, is > > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From dave at aprsfl.net Sun Feb 3 12:13:25 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 13:13:25 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <61866A6C36E249A2A260E5B007FFF219@Inspiron> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <003b01c8664b$ed588180$c8098480$@org> <02b601c86678$d7035e50$850a1af0$@net> <61866A6C36E249A2A260E5B007FFF219@Inspiron> Message-ID: <032101c86690$77f5f270$67e1d750$@net> Hi Karl, Drop me a private e-mail at dave at aprsfl.net and I'll attach the last draft from March of last year to you. I'll highlight the 2 points that were "in debate" at the time highlighted, so you know they are not a set policy right now. The technical aspects of what a server responds with, packet format, etc is 100% accurate though. I'm glad to hear you are responsive to what you hear in this forum. I only wish some of the other developers of some of the larger commercial apps participated here. Your format for polling with the list is -exactly- how we recommend doing it in the whitepaper (we now also have fourth.aprs.net), so you're right on there! The whitepaper has all the details on packet format out of the APRS specification document too. Seeya, Dave -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Karl Uppiano Sent: Sunday, February 03, 2008 12:57 PM To: 'Discussion of weather data quality issues' Subject: Re: [wxqc] CWOP Server: Reporting Constraints Dave, I would be very interested in having a comprehensive and authoritative specification for APRS-CWOP clients as well. I am the developer of WxService (http://mysite.verizon.net/Karl_Uppiano/wxservice.html). I have changed my implementation several times over the past year or two in response to reading updated CWOP web pages, and/or reinterpreting existing pages. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From m.c.crockett at roadrunner.com Sun Feb 3 13:18:17 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 11:18:17 -0800 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> Message-ID: <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> On 03 Feb 2008, at 05:10:34, Steve Dimse wrote: > > On Feb 2, 2008, at 7:02 PM, Merton Campbell Crockett wrote: > >> I had seen the note about using rotate.aprs.net and that it was >> configured with 4 A records. As I have a BIND name server running on >> my system and have only one application, wvcwopd, that queries for >> the >> name server, I would have expected to see each entry going to the >> next >> server in the list. > > If I understand you correctly, you think your your bind would cache > the name and then rotate through it? That may be right if the queries > happen before the expiration time, say every 5 seconds. When a query > occurs after the expiration time (60 seconds for rotate.aprs.net), > your bind hits the aprs.net bind and relays through the order aprs.net > returns. Yes, that is what will happen until the TTL expires. Assuming that I receive the following list from NS3.APRS.NET FIRST CORE-2 THIRD FOURTH the list will be in the following order after the wvcwopd request is satisfied CORE-2 THIRD FOURTH FIRST Were the TTL large, wvcwopd would cycle through the list of servers; however it isn't. In simplistic terms, the probability that my name server will get the same ordered list on its next query is 25 percent. There is a 75 percent probability that wvcwopd will not use CORE-2 for its next connection. This thread was started as a result of a user wondering why his CWOP Client didn't use all of the CWOP Servers or tended to use one server. My 60 percent figure came from "eyeballing" the raw APRS logs for my station. If the goal is to distribute a given client's activity across all of the servers, it isn't going to happen with a small TTL. A small TTL increases the probability that successive client connections will be to the same server. There is a non-zero probability that a given client could always use one server. Merton Campbell Crockett m.c.crockett at roadrunner.com From steve at dimse.com Sun Feb 3 13:33:37 2008 From: steve at dimse.com (Steve Dimse) Date: Sun, 3 Feb 2008 14:33:37 -0500 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> Message-ID: <69184EBB-AF93-4C4A-B817-68C4CD7FFED8@dimse.com> On Feb 3, 2008, at 2:18 PM, Merton Campbell Crockett wrote: > If the goal is to distribute a given client's activity across all of > the servers, it isn't going to happen with a small TTL. A small TTL > increases the probability that successive client connections will be > to the same server. There is a non-zero probability that a given > client could always use one server. > That is the case for people running local bind and using it for their own name service. Very few people fall into that category. You are correct that a very long TTL for these people would take the percentage of consecutive hits below 25%. However, it would raise the frequency well above 25% for the common case. A low TTL essentially makes each report a random event. There, you are right that the chance of a person using a single server is non-zero, but for one day of 10 minute packets, the odds are one in 497323236409786642155382248146820840100456150797347717440463976893159497012533375533056 ! The odds of just using two of the four servers for one day are 1 in 22300745198530623141535718272648361505980416. Non-zero, but very unlikely. Steve From dave at aprsfl.net Sun Feb 3 13:40:11 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 14:40:11 -0500 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> Message-ID: <035e01c8669c$96ff3c60$c4fdb520$@net> There were other mitigating issues that required the TTL to be low. We needed to have the ability to quickly take a server out of rotation, or add another, with almost instantaneous results. There is also the issue of some broadband routers running a "dumb" DNS cache that follows TTL, but doesn't look for more than one A record every on a lookup. (I'll not bash any brand here, but there are several) So I understand your point, it's the odds game. I just I'll be the first to admit with a TTL low, your odds of getting the same IP again is higher than if it was longer, but there are 7/8 more ham stations on APRS-IS than there is CWOP, and many-many stations use this rotation system with decent luck. I've never put a percentage on it, but it's been very effective, I'd say your odds of getting the same IP are about 35-40% based on ham stations that I have looked at with software doing it right, and that odd is something I can live with. Regards, Dave ---- If the goal is to distribute a given client's activity across all of the servers, it isn't going to happen with a small TTL. A small TTL increases the probability that successive client connections will be to the same server. There is a non-zero probability that a given client could always use one server. -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From m.c.crockett at roadrunner.com Sun Feb 3 16:31:18 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 14:31:18 -0800 Subject: [wxqc] How rotate.aprs.net is supposed to work In-Reply-To: <035e01c8669c$96ff3c60$c4fdb520$@net> References: <968E6C97-673F-44BD-BB97-7F4232F0CEC1@roadrunner.com> <9191B36B-6DED-406E-A606-C0C977BB4395@roadrunner.com> <88ABE419-D787-4C64-AFF7-73BBC4FE25E2@dimse.com> <9C12D93A-B05F-4301-81B3-154FAA57AE95@roadrunner.com> <035e01c8669c$96ff3c60$c4fdb520$@net> Message-ID: On 03 Feb 2008, at 11:40:11, Dave Anderson KG4YZY wrote: > So I understand your point, it's the odds game. I just I'll be the > first to > admit with a TTL low, your odds of getting the same IP again is > higher than > if it was longer, but there are 7/8 more ham stations on APRS-IS > than there > is CWOP, and many-many stations use this rotation system with decent > luck. > I've never put a percentage on it, but it's been very effective, I'd > say > your odds of getting the same IP are about 35-40% based on ham > stations that > I have looked at with software doing it right, and that odd is > something I > can live with. Out of morbid curiosity, I took a snap shot of today's APRS activity. The distribution wasn't as skewed as the last time that I looked. The distribution was the following. FIRST 2.3% Core-2 31.7% THIRD 27.9% FOURTH 37.9% In addition, I connected to the same server that I used for the previous connection 34 percent of the time. As long as the load on each server is balanced, it really doesn't matter if a client rarely connects to one or two of the servers. Merton Campbell Crockett m.c.crockett at roadrunner.com From m.c.crockett at roadrunner.com Sun Feb 3 16:47:28 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 14:47:28 -0800 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <0D422FD4-C1AD-4773-96BB-FD7B87B5C592@dimse.com> References: <10CAC54E-3F47-4555-AAB2-835DE80E8BA5@roadrunner.com> <01e801c8661c$9fb3d760$df1b8620$@net> <3199AA8D-8FA7-4623-8EEB-4859C797F5C6@dimse.com> <7E1BBD57-F913-45D1-9672-0C14DB0B2B6E@roadrunner.com> <0D422FD4-C1AD-4773-96BB-FD7B87B5C592@dimse.com> Message-ID: <7019C629-0D74-41C1-98DA-5645625217A0@roadrunner.com> On 03 Feb 2008, at 08:31:55, Steve Dimse wrote: > > On Feb 3, 2008, at 11:18 AM, Merton Campbell Crockett wrote: > >> Assuming that CW9999 connects at 16 minutes after the hour, can it >> send the 3 reports for the periods ending at 05, 10, and 15 minutes >> after the hour? The catch is that the 10 and 15 minute reports >> cannot >> be discarded as was suggested by one of the respondents to my initial >> question. >> > No. The APRS Internet System is a realtime system. There is no > provision for sending old data, timestamps in the packet are ignored. > In the example above, one minute isn't going to make a difference, but > if you have access to the data at 16 minutes after the hour that is > what should be sent, not the 15 minute data. So, there is a restriction. One can only send a single report per connection. As a result, if the weather station hardware generates a report at 5 minute intervals, the process responsible for sending reports to CWOP must send them at 5 minute intervals. The process would be responsible for adjusting the report were it to transmit data to CWOP at a 15 minute interval, i.e. wind gust, precipitation, etc. Merton Campbell Crockett m.c.crockett at roadrunner.com From m.c.crockett at roadrunner.com Sun Feb 3 17:06:09 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 15:06:09 -0800 Subject: [wxqc] Weather Station Location In-Reply-To: <8E1C0561F8EE4E31B5C6C42C6DCFAEC1@sauron> References: <8E1C0561F8EE4E31B5C6C42C6DCFAEC1@sauron> Message-ID: On 02 Feb 2008, at 18:23:19, Keith Miller wrote: >> >> I used the "sea-level equivalent" barometric pressure instead >> of the altimeter setting at the Pt Mugu, Naval Air Warfare >> Center. I've re-adjusted my barometer based on the latter >> and MADIS is much happier with the results: two thumbs up. >> > > Ah, that would make a difference. Did you adjust to what's > being displayed on the console, or what is being sent to CWOP? > The console displays sea-level reduced pressure... I used the Davis Instruments procedure for adjusting the Vantage Pro2 console. The wview 3.4.1 that I'm using doesn't appear to have a way of adjusting the data after it is retrieved. I have only looked at the wvcwopd daemon source code. It could be adjusted by one of the other daemons. Merton Campbell Crockett m.c.crockett at roadrunner.com From m.c.crockett at roadrunner.com Sun Feb 3 17:13:56 2008 From: m.c.crockett at roadrunner.com (Merton Campbell Crockett) Date: Sun, 3 Feb 2008 15:13:56 -0800 Subject: [wxqc] Precipitation Data and Quality Control Message-ID: Why doesn't my precipitation data appear in either the NOAA or MesoWest reports for my weather station? This data appears for other CWOP sites but not all. Also, the Quality Control column exists on the NOAA reports but is blank. I thought that I saw an OK appearing in the column when I first started transmitting data. Merton Campbell Crockett m.c.crockett at roadrunner.com From dave at aprsfl.net Sun Feb 3 19:36:46 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 20:36:46 -0500 Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP. Message-ID: <043301c866ce$67c8d410$375a7c30$@net> Hello all, I am here to regretfully announce that the third.aprs.net and fourth.aprs.net servers that I personally own and operate will no longer be supporting the CWOP program. Allow me to give a brief history on events, so you understand my decision. I am a ham radio operator. The APRS-IS network was built by hams, for hams. CWOP, non ham users, are guests on our network to get to MADIS. CWOP users account for 1/8 of the total station count on APRS, the other 7/8 are non weather ham radio users. The traffic is disproportionate though, as CWOP users take up a full ? of the bandwidth passing thru the servers. Back January of last year, the previous group of server sysops that had ran the servers for CWOP since day one abruptly pulled support for the CWOP program one Friday. No excuse was given beyond that it was in the best interest of both parties. This was a devastating blow to CWOP. The core group of server sysops were contacted about this, and we spung into action to help get CWOP taken care of. It required extensive work to get this to happen, as we had only three servers at the time. In less than 48 hours we managed to get a 4th server online, and had started moving users over. Shortly after this, I observed that the method that the stations were using to submit the data was horribly wrong for the APRS-IS network (does not follow the spec), knowing that as the load increased, we would have problems. I started writing a white paper, which I have sent to several of you on request, that outlined everything a developer needed to know to write a program to use the APRS-IS network without causing undue network load. During the development of this white paper there were 8 if I recall folks that were involved with the ?fine tuning? of it. At the time, we wanted to set the host name for CWOP users to use as cwop.aprs.net, so later we could move you guys to your own dedicated network of servers (thinking ahead). There were also a few other items that we wanted addressed, such as the polling time, and polling interval. Dave Helms, one of the CWOP administrators refused to accept some of the items in the draft. After several attempts to talk reason into him, I just said the heck with it, and let it sit in my temp directory, never finished. No one else picked up the ropes to keep going, either, and everyone had the last version of the draft! Here?s the problems The core sysops have been begging Dave to get the developers to fix their software. Dave, fearing that developers would pull out of the CWOP program with the changes the core was wanting to be made, has had very little done about this. When most of you have problems with data loss, or difficulty in connecting to a server, it?s because the policies that the core put in place were never implemented by most of the software developers because Dave never passed it on to them. We can only operate a network so long as the software ?you- use is set to work with it correctly. So a year later, after suggestions have been made, most of you are still using software that is broken, or have incorrect settings in place to work 100% correctly with the servers. And we?ve been asking over, and over, and over for these changes to be implemented, as even as far back as October we started noticing a detrimental impact on the network, but nothing has been done. Jump to today, here?s the problems that still exist that are causing server sysops problems and are causing you problems due to the fact that most of the CWOP programs don?t follow the rules: The lion share of you have software that uses your PC clock for polling time, meaning our servers have to accept thousands of stations every 5 minutes then sit relatively idle for 4 more minutes. This with the addition of new Christmas gift stations this year has been causing countless crashes, data loss, and other issues that the server sysops have been working on. Each server can only accept 50 backlogged connections at a shot, and frequently port 14580 way exceeds that. So it?s caused instabilities in the network, as networking queue times go very high when this happens affecting HAMS as well as CWOP. We?ve asked this to be fixed, and heard silence. Polling interval is another issue. MADIS takes the data from the APRS-IS network via Findu only every 15 minutes, but we still have stations that were sending data in every minute. Keep in mind, you ?don?t stay- connected to the server, you connect, drop off, and disconnect. To Russ Chadwick?s credit action was finally taken just here recently about these stations, and we?ve banned them at the server level, but that still doesn?t prevent them from going to connect, and tying up a server slot every minute. 5 minutes is the standard that amateur radio operators have used on APRS for probably 10 years now. Why that wasn?t just hard coded in programs, I don?t know, but again, this is another issue we?ve been asking over and over to get fixed, and it yet is. Round Robin DNS still doesn?t work for the most of you. Over ? of the CWOP users use third alone. Many of the programs simply only look up the IP address of the rotate.aprs.net when you first load the program or connect the first time, and then it ?never- goes back to ask if the IP has changed, or check if there is another one for load balancing. Again, asked over and over for this to be addresses with the developers, still hasn?t been. I am by no means saying that every piece of CWOP software is broken!!! Numerous apps ?do it right- as the developers of those programs have been in this forum, seen some of our technical posts about problems and corrected their code. But the amount of developers who seem to care about doing it right is very few, and the largest CWOP applications that the bulk of the users use, simply do not follow the rules. So who suffers? We all do. As the server sysops, we have to sit, being sucker punched every 5 minutes right now. You do as not all of your weather data makes it thru. APRS-IS is the internet side of a VHF radio system on 144.390mhz that does packet radio. Weather is only a small fraction of APRS?s overall functionality, so as system operators we have to be sensitive to this, not to allow CWOP to cause harm to the amateur radio operators that rely on this for emergency communications and safety of life traffic. The servers are ran by a 100% volunteer group of system operators that in the case of the core servers, are all ran in datacenters. Fourth alone is a 8 core Xeon box running VMWare ESX server with Windows 2003 running the fourth virtual machine on that box. I spent $4500.00 for that server to ?SUPPORT- cwop. I also donate about 6 meg of tier1 quality backbone bandwidth every day to this program. Unlike Findu, where Steve gets support from his users of that website, the server sysops, who without there would be no network to use, we get zero support from the user community. That? after thousands of dollars in hardware and hundreds of hours of time. So how was I repaid for my generosity???? ? Requests to make changes to the client software went unanswered, allowing things to spiral to where they are now. ? Every time anything happens to the network, the server sysops are automatically crucified for doing something wrong. ? I can?t think of the last time anyone stopped to tell me how much they appreciate what I do. So it all boils down to this morning, when I posed the message at 10:24 am to Seth about the standing of the software problems. Shortly afterward, I go this from Dave Helms: Dave, I, too, have absolutely no interest in revisiting the events of January-March 2007 (publicly or otherwise). I am willing to support the notion of segmenting CW traffic to improve stability of APRS-IS, to include developing a plan for implementing the cwop.aprs.net round robin server name. In last year's crisis mode, I agreed to contact virtually all CW and Ham weather station operators to facilitate the switch from Tier2 servers to Core servers included on the rotate.aprs.net RR server domain. In the future, I will rely on encouraging individuals to read the CWOP News Page to advertise the RR server name change, and through routine client application upgrades rather than making individual email contacts. As we move forward, I request one thing... that we not make the "CW' (non-Ham) volunteers second class users. Their contributions are equally valuable to NOAA as validated Ham Radio users who send weather reports to APRS-IS and NOAA. If weather traffic must be (temporarily) throttled back to preserve APRS-IS stability, then all weather contributions should be equally reduced, Ham and non-Hams alike. Going forward, we need to find a way to add capacity to the APRS-IS to take the load of the expected growing number of users. Hence, we need to develop an architecture that will allow exploitation of both Ham Radio APRS-IS server resources and non-Ham servers which may be available. These non-Ham servers and bandwidth may be donated from government, universities, and private sector groups. Regards, Dave Finally after a year, Dave concedes to allowing us to start using CWOP.APRS.NET. Well the problem is that it?s a day late, and a dollar short. He further more says it?ll only be a news post on a website, and for when folks update software. Well if the last change to rotate is any indication of how well that will go, there still are 800 of you using the old servers to this DAY. So if he?s not going to help promote it, we?ll have a network that is in shambles here in the next 90 days due to load. I just don?t think Dave understands the gravity of the problem. Then the straw that broke the camel?s back was in the second paragraph. Insisting that ham stations also be reduced to that of non hams. Well first off, ham stations are generally sharing a very limited amount of bandwidth on 144.390mhz (1200bps). From peer pressure alone, no ham will beacon weather more than every 5 minutes without hearing about it from other local hams. We have a system in place that works well, using nothing but peer pressure. The fact that he?s insisting hams back off, on their own network, that was built by hams, really offended me, and so far most of the hams? I?ve shared this with seem to agree. Dave and the NWS pay NOTHING for what we do on the network. How dare he tell us what to do on our own network. CWOP are GUESTS on our network. Dave has in the past shown callous towards the hams, never this blatant, but I?ve seen it in other messages. This disgusts me. This would be no different than me coming into your house as a guest and telling you can only spend 5 hours a day watching TV. So I write Dave back a ?very- long message, and at this point I?m very upset about this. I lay blame for the problems we have right now where it needs to be placed, squarely on his shoulders. He was the reason the white paper was given up on, as if it wasn?t Dave?s way, it was no good. Yeah, I?ll admit my message to him was long, and rather blunt. I was direct with him, and factually telling him it?s come to a point where change has to occur, or the network will change around him. Well, here was his response to that: Dave, In the time your spent wiring this email, you would could have instead been making progress on the a white paper your said you would complete last year. Don't blame the client developers for something you knew that was needed over a year ago. Dave Helms Accusing me of not finishing a document that he opposed?!!?!?!?!?!! Excuse me? Draft 3 was what would have gotten published had he not objected to the hostname. The rest ?is- done right now. Well if the NWS employee that is supposed to be our liaison to the developers is taking this attitude towards me, with the thousands I have spent to support CWOP, I hereby pull my official support from the program. I realize this will be a heavy hit for CWOP, as I take a full one half of the server capacity with me. I am greatly sorry for that. The last people I wanted to hurt were the ones I was out to help, but with Dave Helms at the ?Helm?, it?s over for me. I cannot continue to be badgered, beaten, ignored, then finally blamed for it to begin with over a volunteer operation. If any of you would like to talk about this more, I encourage you to send me a private e-mail at dave at aprsfl.net. I?m sure this message will get me banned on this forum, so I?ll not be checking back. I?m glad I?ve been able to help up till now, and I really am truly sorry for going out this way, but I just can?t go on with this type of pervasive attitude . Let alone one that is so callous towards the folks that this network was built for, ham radio operators. Regards, Dave Anderson KG4YZY -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080203/48e5867b/attachment-0001.html From dave at aprsfl.net Sun Feb 3 19:37:52 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 20:37:52 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <020320082322.8692.47A64CCE000AAF43000021F422058861729C03040A089C0B@comcast.net> References: <020320082322.8692.47A64CCE000AAF43000021F422058861729C03040A089C0B@comcast.net> Message-ID: <043e01c866ce$8ebf58a0$ac3e09e0$@net> Dave, The third draft of the whitepaper is -exactly- how I would have submitted it back in April of last year, however you would not take it due to cwop.aprs.net being the hostname. Don't you -DARE- blame me about your lack of willingness to "upset the developers", it was you who refused to let that document pass with cwop.aprs.net in it! I can't believe how callous you are towards the 3 people who run the servers required to make the CWOP Citizen Weather Observing Program work. Without us, the program doesn't exist. We do what we do totally free on a volunteer basis, and this is how we're repaid? As a NWS employee, you really are putting a bad light on your employer on this, as when this goes public tonight, it'll be NOAA with a black eye, not me. I busted my butt for you and Russ back when Phil with Tier2 told you to go fly a kite last year. I wasted an entire WEEK of my time helping CWOP users move to our servers. Based on these last two e-mails you have sent me, I actually see -why- Phil pulled support. I spent money buying a $4500.00 new server that became the fourth server to SUPPORT the CWOP load, and I can't even think of the amount of time I've put into this program. I wanted to work this out, I have been BEGGING you on 3 key points for the past 90 days be passed on to developers and NOTHING has happened with that. You have put me in a position with Sean N4SCF that leaves me no option here, he does own 1/2 the equipment that runs my servers, as he, as a 2005 NOAA Environment Hero Award recipient he will not stand for the NWS walking on amateur radio operators the way you suggest we do. As of now third and fourth are no longer in rotate, so good luck with that, you just lost 1/2 your servers for CWOP access. I'll also be posting your callous attitude about "second class" regarding hams in several forums tonight, and be sending it off to several folks at the NWS that I am sure will find your attitude towards a volunteer network as offensive as I find it. I've already shared your message you sent me earlier today with a few fellow hams that are neutral (nothing to do with APRS), and I tell ya, non appreciated what you said, so it's not just me reading it wrong. Regards, Dave Anderson KG4YZY Sysop Third.aprs.net & fourth.aprs.net -----Original Message----- From: dshelms at comcast.net [mailto:dshelms at comcast.net] Sent: Sunday, February 03, 2008 6:23 PM To: Dave Anderson KG4YZY Cc: 'Greg WB6ZSU'; 'Gerry Creager'; 'Pete Loveall AE5PL'; 'Steve Dimse'; Russell.B.Chadwick at noaa.gov; 'Sean Fleeman' Subject: RE: Re: [wxqc] CWOP Server: Reporting Constraints Dave, In the time your spent wiring this email, you would could have instead been making progress on the a white paper your said you would complete last year. Don't blame the client developers for something you knew that was needed over a year ago. Dave Helms -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From dave at aprsfl.net Sun Feb 3 19:40:41 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 20:40:41 -0500 Subject: [wxqc] CWOP Server: Reporting Constraints In-Reply-To: <020320082322.8692.47A64CCE000AAF43000021F422058861729C03040A089C0B@comcast.net> References: <020320082322.8692.47A64CCE000AAF43000021F422058861729C03040A089C0B@comcast.net> Message-ID: <044801c866ce$f3c66220$db532660$@net> This is the last correspondence that ultimately resulted in my pulling support from the program. ------------------------ Dave, The third draft of the whitepaper is -exactly- how I would have submitted it back in April of last year, however you would not take it due to cwop.aprs.net being the hostname. Don't you -DARE- blame me about your lack of willingness to "upset the developers", it was you who refused to let that document pass with cwop.aprs.net in it! I can't believe how callous you are towards the 3 people who run the servers required to make the CWOP Citizen Weather Observing Program work. Without us, the program doesn't exist. We do what we do totally free on a volunteer basis, and this is how we're repaid? As a NWS employee, you really are putting a bad light on your employer on this, as when this goes public tonight, it'll be NOAA with a black eye, not me. I busted my butt for you and Russ back when Phil with Tier2 told you to go fly a kite last year. I wasted an entire WEEK of my time helping CWOP users move to our servers. Based on these last two e-mails you have sent me, I actually see -why- Phil pulled support. I spent money buying a $4500.00 new server that became the fourth server to SUPPORT the CWOP load, and I can't even think of the amount of time I've put into this program. I wanted to work this out, I have been BEGGING you on 3 key points for the past 90 days be passed on to developers and NOTHING has happened with that. You have put me in a position with Sean N4SCF that leaves me no option here, he does own 1/2 the equipment that runs my servers, as he, as a 2005 NOAA Environment Hero Award recipient he will not stand for the NWS walking on amateur radio operators the way you suggest we do. As of now third and fourth are no longer in rotate, so good luck with that, you just lost 1/2 your servers for CWOP access. I'll also be posting your callous attitude about "second class" regarding hams in several forums tonight, and be sending it off to several folks at the NWS that I am sure will find your attitude towards a volunteer network as offensive as I find it. I've already shared your message you sent me earlier today with a few fellow hams that are neutral (nothing to do with APRS), and I tell ya, non appreciated what you said, so it's not just me reading it wrong. Regards, Dave Anderson KG4YZY Sysop Third.aprs.net & fourth.aprs.net -----Original Message----- From: dshelms at comcast.net [mailto:dshelms at comcast.net] Sent: Sunday, February 03, 2008 6:23 PM To: Dave Anderson KG4YZY Cc: 'Greg WB6ZSU'; 'Gerry Creager'; 'Pete Loveall AE5PL'; 'Steve Dimse'; Russell.B.Chadwick at noaa.gov; 'Sean Fleeman' Subject: RE: Re: [wxqc] CWOP Server: Reporting Constraints Dave, In the time your spent wiring this email, you would could have instead been making progress on the a white paper your said you would complete last year. Don't blame the client developers for something you knew that was needed over a year ago. Dave Helms -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From dave at aprsfl.net Sun Feb 3 19:56:29 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Sun, 3 Feb 2008 20:56:29 -0500 Subject: [wxqc] One last thought and I am tuning out for now... Message-ID: <044c01c866d1$28558370$79008a50$@net> There was recently a discussion about adding more servers to solve this problem with the load.. That will not solve the problem. This is a packet per second load problem. With APRS, all the servers are interconnected, so heavy load on one server traffic wise is felt by all of them. If more servers were to have been added to the core, it would have solved nothing. Keep in mind your weather packet not only goes to findu, but to EVERY ham radio user connected to the network. So if third has 250 hams connected, every CWOP packet has to be send to all of those users. See how crazy out of control this can get when everyone goes to hit the server at the same time? Again, to be clear, additional server will not solve this. Changing the polling "time" to be random and correctly using rotate load balancing -will-, but the network CANNOT fix these problems. Regards, Dave KG4YZY -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080203/09546580/attachment.html From fcorey at monticelloweather.com Sun Feb 3 20:00:03 2008 From: fcorey at monticelloweather.com (Fred Corey) Date: Sun, 3 Feb 2008 21:00:03 -0500 Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP. References: <043301c866ce$67c8d410$375a7c30$@net> Message-ID: <013d01c866d1$aa383ef0$9865fea9@D1GGTN71> Dave Helms, please help!!!! We need CWOP. Turn the other cheek and fix things!!!! We all need you support. Sincerely, Fred Corey. ----- Original Message ----- From: Dave Anderson KG4YZY To: 'Discussion of weather data quality issues' Sent: Sunday, February 03, 2008 8:36 PM Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP. Hello all, I am here to regretfully announce that the third.aprs.net and fourth.aprs.net servers that I personally own and operate will no longer be supporting the CWOP program. Allow me to give a brief history on events, so you understand my decision. I am a ham radio operator. The APRS-IS network was built by hams, for hams. CWOP, non ham users, are guests on our network to get to MADIS. CWOP users account for 1/8 of the total station count on APRS, the other 7/8 are non weather ham radio users. The traffic is disproportionate though, as CWOP users take up a full ? of the bandwidth passing thru the servers. Back January of last year, the previous group of server sysops that had ran the servers for CWOP since day one abruptly pulled support for the CWOP program one Friday. No excuse was given beyond that it was in the best interest of both parties. This was a devastating blow to CWOP. The core group of server sysops were contacted about this, and we spung into action to help get CWOP taken care of. It required extensive work to get this to happen, as we had only three servers at the time. In less than 48 hours we managed to get a 4th server online, and had started moving users over. Shortly after this, I observed that the method that the stations were using to submit the data was horribly wrong for the APRS-IS network (does not follow the spec), knowing that as the load increased, we would have problems. I started writing a white paper, which I have sent to several of you on request, that outlined everything a developer needed to know to write a program to use the APRS-IS network without causing undue network load. During the development of this white paper there were 8 if I recall folks that were involved with the "fine tuning" of it. At the time, we wanted to set the host name for CWOP users to use as cwop.aprs.net, so later we could move you guys to your own dedicated network of servers (thinking ahead). There were also a few other items that we wanted addressed, such as the polling time, and polling interval. Dave Helms, one of the CWOP administrators refused to accept some of the items in the draft. After several attempts to talk reason into him, I just said the heck with it, and let it sit in my temp directory, never finished. No one else picked up the ropes to keep going, either, and everyone had the last version of the draft! Here's the problems. The core sysops have been begging Dave to get the developers to fix their software. Dave, fearing that developers would pull out of the CWOP program with the changes the core was wanting to be made, has had very little done about this. When most of you have problems with data loss, or difficulty in connecting to a server, it's because the policies that the core put in place were never implemented by most of the software developers because Dave never passed it on to them. We can only operate a network so long as the software -you- use is set to work with it correctly. So a year later, after suggestions have been made, most of you are still using software that is broken, or have incorrect settings in place to work 100% correctly with the servers. And we've been asking over, and over, and over for these changes to be implemented, as even as far back as October we started noticing a detrimental impact on the network, but nothing has been done. Jump to today, here's the problems that still exist that are causing server sysops problems and are causing you problems due to the fact that most of the CWOP programs don't follow the rules: The lion share of you have software that uses your PC clock for polling time, meaning our servers have to accept thousands of stations every 5 minutes then sit relatively idle for 4 more minutes. This with the addition of new Christmas gift stations this year has been causing countless crashes, data loss, and other issues that the server sysops have been working on. Each server can only accept 50 backlogged connections at a shot, and frequently port 14580 way exceeds that. So it's caused instabilities in the network, as networking queue times go very high when this happens affecting HAMS as well as CWOP. We've asked this to be fixed, and heard silence. Polling interval is another issue. MADIS takes the data from the APRS-IS network via Findu only every 15 minutes, but we still have stations that were sending data in every minute. Keep in mind, you -don't stay- connected to the server, you connect, drop off, and disconnect. To Russ Chadwick's credit action was finally taken just here recently about these stations, and we've banned them at the server level, but that still doesn't prevent them from going to connect, and tying up a server slot every minute. 5 minutes is the standard that amateur radio operators have used on APRS for probably 10 years now. Why that wasn't just hard coded in programs, I don't know, but again, this is another issue we've been asking over and over to get fixed, and it yet is. Round Robin DNS still doesn't work for the most of you. Over ? of the CWOP users use third alone. Many of the programs simply only look up the IP address of the rotate.aprs.net when you first load the program or connect the first time, and then it -never- goes back to ask if the IP has changed, or check if there is another one for load balancing. Again, asked over and over for this to be addresses with the developers, still hasn't been. I am by no means saying that every piece of CWOP software is broken!!! Numerous apps -do it right- as the developers of those programs have been in this forum, seen some of our technical posts about problems and corrected their code. But the amount of developers who seem to care about doing it right is very few, and the largest CWOP applications that the bulk of the users use, simply do not follow the rules. So who suffers? We all do. As the server sysops, we have to sit, being sucker punched every 5 minutes right now. You do as not all of your weather data makes it thru. APRS-IS is the internet side of a VHF radio system on 144.390mhz that does packet radio. Weather is only a small fraction of APRS's overall functionality, so as system operators we have to be sensitive to this, not to allow CWOP to cause harm to the amateur radio operators that rely on this for emergency communications and safety of life traffic. The servers are ran by a 100% volunteer group of system operators that in the case of the core servers, are all ran in datacenters. Fourth alone is a 8 core Xeon box running VMWare ESX server with Windows 2003 running the fourth virtual machine on that box. I spent $4500.00 for that server to -SUPPORT- cwop. I also donate about 6 meg of tier1 quality backbone bandwidth every day to this program. Unlike Findu, where Steve gets support from his users of that website, the server sysops, who without there would be no network to use, we get zero support from the user community. That' after thousands of dollars in hardware and hundreds of hours of time. So how was I repaid for my generosity???? ? Requests to make changes to the client software went unanswered, allowing things to spiral to where they are now. ? Every time anything happens to the network, the server sysops are automatically crucified for doing something wrong. ? I can't think of the last time anyone stopped to tell me how much they appreciate what I do. So it all boils down to this morning, when I posed the message at 10:24 am to Seth about the standing of the software problems. Shortly afterward, I go this from Dave Helms: Dave, I, too, have absolutely no interest in revisiting the events of January-March 2007 (publicly or otherwise). I am willing to support the notion of segmenting CW traffic to improve stability of APRS-IS, to include developing a plan for implementing the cwop.aprs.net round robin server name. In last year's crisis mode, I agreed to contact virtually all CW and Ham weather station operators to facilitate the switch from Tier2 servers to Core servers included on the rotate.aprs.net RR server domain. In the future, I will rely on encouraging individuals to read the CWOP News Page to advertise the RR server name change, and through routine client application upgrades rather than making individual email contacts. As we move forward, I request one thing... that we not make the "CW' (non-Ham) volunteers second class users. Their contributions are equally valuable to NOAA as validated Ham Radio users who send weather reports to APRS-IS and NOAA. If weather traffic must be (temporarily) throttled back to preserve APRS-IS stability, then all weather contributions should be equally reduced, Ham and non-Hams alike. Going forward, we need to find a way to add capacity to the APRS-IS to take the load of the expected growing number of users. Hence, we need to develop an architecture that will allow exploitation of both Ham Radio APRS-IS server resources and non-Ham servers which may be available. These non-Ham servers and bandwidth may be donated from government, universities, and private sector groups. Regards, Dave Finally after a year, Dave concedes to allowing us to start using CWOP.APRS.NET. Well the problem is that it's a day late, and a dollar short. He further more says it'll only be a news post on a website, and for when folks update software. Well if the last change to rotate is any indication of how well that will go, there still are 800 of you using the old servers to this DAY. So if he's not going to help promote it, we'll have a network that is in shambles here in the next 90 days due to load. I just don't think Dave understands the gravity of the problem. Then the straw that broke the camel's back was in the second paragraph. Insisting that ham stations also be reduced to that of non hams. Well first off, ham stations are generally sharing a very limited amount of bandwidth on 144.390mhz (1200bps). From peer pressure alone, no ham will beacon weather more than every 5 minutes without hearing about it from other local hams. We have a system in place that works well, using nothing but peer pressure. The fact that he's insisting hams back off, on their own network, that was built by hams, really offended me, and so far most of the hams' I've shared this with seem to agree. Dave and the NWS pay NOTHING for what we do on the network. How dare he tell us what to do on our own network. CWOP are GUESTS on our network. Dave has in the past shown callous towards the hams, never this blatant, but I've seen it in other messages. This disgusts me. This would be no different than me coming into your house as a guest and telling you can only spend 5 hours a day watching TV. So I write Dave back a -very- long message, and at this point I'm very upset about this. I lay blame for the problems we have right now where it needs to be placed, squarely on his shoulders. He was the reason the white paper was given up on, as if it wasn't Dave's way, it was no good. Yeah, I'll admit my message to him was long, and rather blunt. I was direct with him, and factually telling him it's come to a point where change has to occur, or the network will change around him. Well, here was his response to that: Dave, In the time your spent wiring this email, you would could have instead been making progress on the a white paper your said you would complete last year. Don't blame the client developers for something you knew that was needed over a year ago. Dave Helms Accusing me of not finishing a document that he opposed?!!?!?!?!?!! Excuse me? Draft 3 was what would have gotten published had he not objected to the hostname. The rest -is- done right now. Well if the NWS employee that is supposed to be our liaison to the developers is taking this attitude towards me, with the thousands I have spent to support CWOP, I hereby pull my official support from the program. I realize this will be a heavy hit for CWOP, as I take a full one half of the server capacity with me. I am greatly sorry for that. The last people I wanted to hurt were the ones I was out to help, but with Dave Helms at the "Helm", it's over for me. I cannot continue to be badgered, beaten, ignored, then finally blamed for it to begin with over a volunteer operation. If any of you would like to talk about this more, I encourage you to send me a private e-mail at dave at aprsfl.net. I'm sure this message will get me banned on this forum, so I'll not be checking back. I'm glad I've been able to help up till now, and I really am truly sorry for going out this way, but I just can't go on with this type of pervasive attitude.. Let alone one that is so callous towards the folks that this network was built for, ham radio operators. Regards, Dave Anderson KG4YZY -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ------------------------------------------------------------------------------ _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080203/29f89948/attachment-0001.html From dshelms at comcast.net Sun Feb 3 21:44:05 2008 From: dshelms at comcast.net (dshelms at comcast.net) Date: Mon, 04 Feb 2008 03:44:05 +0000 Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP. Message-ID: <020420080344.17795.47A68A040009C5F10000458322007507449C03040A089C0B@comcast.net> All: Tonight, I exchanged a personal email with Dave Anderson stating I wanted to move forward with him on the cwop.aprs.net effort. This is exactly what DaveA wanted last year, about two months after we have encouraged everyone to move to the Core servers through rotate.aprs.net. Again, last spring, I requested a period of stability in implementing the cwop.aprs.net. With thousands of users and tens of developers, you simply cannot change implemented code very quickly. For what it is worth as of this evening, DaveA and I are in agreement on the course of action to improve the stability of the system. They are: reducing max upload frequency to 10 minutes, providing documentation for the client applications for connecting with the APRS-IS, and developing a segregated instance of APRS-IS for CWOP. I really do appreciate Dave Anderson's efforts to support CWOP/APRSWXNET, including his decision to donate server and bandwidth to the cause. Dave has every right withdraw his support, but I encourage Dave to withdraw his support in a manner that minimizes the impact on the program he, and many others, worked very hard build. Dave Helms CW0351 -------------- Original message ---------------------- From: "Fred Corey" > Dave Helms, please help!!!! We need CWOP. Turn the other cheek and fix > things!!!! > > We all need you support. > > Sincerely, > Fred Corey. > > > ----- Original Message ----- > From: Dave Anderson KG4YZY > To: 'Discussion of weather data quality issues' > Sent: Sunday, February 03, 2008 8:36 PM > Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP. > > > > > Hello all, > > > > I am here to regretfully announce that the third.aprs.net and fourth.aprs.net > servers that I personally own and operate will no longer be supporting the CWOP > program. > > > > Allow me to give a brief history on events, so you understand my decision. > > > > I am a ham radio operator. The APRS-IS network was built by hams, for hams. > CWOP, non ham users, are guests on our network to get to MADIS. CWOP users > account for 1/8 of the total station count on APRS, the other 7/8 are non > weather ham radio users. The traffic is disproportionate though, as CWOP users > take up a full ? of the bandwidth passing thru the servers. > > > > Back January of last year, the previous group of server sysops that had ran > the servers for CWOP since day one abruptly pulled support for the CWOP program > one Friday. No excuse was given beyond that it was in the best interest of both > parties. This was a devastating blow to CWOP. > > > > The core group of server sysops were contacted about this, and we spung into > action to help get CWOP taken care of. It required extensive work to get this > to happen, as we had only three servers at the time. In less than 48 hours we > managed to get a 4th server online, and had started moving users over. > > > > Shortly after this, I observed that the method that the stations were using to > submit the data was horribly wrong for the APRS-IS network (does not follow the > spec), knowing that as the load increased, we would have problems. I started > writing a white paper, which I have sent to several of you on request, that > outlined everything a developer needed to know to write a program to use the > APRS-IS network without causing undue network load. > > > > During the development of this white paper there were 8 if I recall folks that > were involved with the "fine tuning" of it. At the time, we wanted to set the > host name for CWOP users to use as cwop.aprs.net, so later we could move you > guys to your own dedicated network of servers (thinking ahead). There were also > a few other items that we wanted addressed, such as the polling time, and > polling interval. Dave Helms, one of the CWOP administrators refused to > accept some of the items in the draft. After several attempts to talk reason > into him, I just said the heck with it, and let it sit in my temp directory, > never finished. No one else picked up the ropes to keep going, either, and > everyone had the last version of the draft! > > > > Here's the problems. The core sysops have been begging Dave to get the > developers to fix their software. Dave, fearing that developers would pull out > of the CWOP program with the changes the core was wanting to be made, has had > very little done about this. When most of you have problems with data loss, or > difficulty in connecting to a server, it's because the policies that the core > put in place were never implemented by most of the software developers because > Dave never passed it on to them. We can only operate a network so long as the > software -you- use is set to work with it correctly. So a year later, after > suggestions have been made, most of you are still using software that is broken, > or have incorrect settings in place to work 100% correctly with the servers. > And we've been asking over, and over, and over for these changes to be > implemented, as even as far back as October we started noticing a detrimental > impact on the network, but nothing has been done. > > > > Jump to today, here's the problems that still exist that are causing server > sysops problems and are causing you problems due to the fact that most of the > CWOP programs don't follow the rules: > > > > The lion share of you have software that uses your PC clock for polling time, > meaning our servers have to accept thousands of stations every 5 minutes then > sit relatively idle for 4 more minutes. This with the addition of new Christmas > gift stations this year has been causing countless crashes, data loss, and other > issues that the server sysops have been working on. Each server can only > accept 50 backlogged connections at a shot, and frequently port 14580 way > exceeds that. So it's caused instabilities in the network, as networking queue > times go very high when this happens affecting HAMS as well as CWOP. We've > asked this to be fixed, and heard silence. > > > > Polling interval is another issue. MADIS takes the data from the APRS-IS > network via Findu only every 15 minutes, but we still have stations that were > sending data in every minute. Keep in mind, you -don't stay- connected to the > server, you connect, drop off, and disconnect. To Russ Chadwick's credit > action was finally taken just here recently about these stations, and we've > banned them at the server level, but that still doesn't prevent them from going > to connect, and tying up a server slot every minute. 5 minutes is the standard > that amateur radio operators have used on APRS for probably 10 years now. Why > that wasn't just hard coded in programs, I don't know, but again, this is > another issue we've been asking over and over to get fixed, and it yet is. > > > > Round Robin DNS still doesn't work for the most of you. Over ? of the CWOP > users use third alone. Many of the programs simply only look up the IP address > of the rotate.aprs.net when you first load the program or connect the first > time, and then it -never- goes back to ask if the IP has changed, or check if > there is another one for load balancing. Again, asked over and over for this to > be addresses with the developers, still hasn't been. > > > > I am by no means saying that every piece of CWOP software is broken!!! > Numerous apps -do it right- as the developers of those programs have been in > this forum, seen some of our technical posts about problems and corrected their > code. But the amount of developers who seem to care about doing it right is > very few, and the largest CWOP applications that the bulk of the users use, > simply do not follow the rules. > > > > So who suffers? We all do. As the server sysops, we have to sit, being > sucker punched every 5 minutes right now. You do as not all of your weather > data makes it thru. > > > > APRS-IS is the internet side of a VHF radio system on 144.390mhz that does > packet radio. Weather is only a small fraction of APRS's overall functionality, > so as system operators we have to be sensitive to this, not to allow CWOP to > cause harm to the amateur radio operators that rely on this for emergency > communications and safety of life traffic. > > > > The servers are ran by a 100% volunteer group of system operators that in the > case of the core servers, are all ran in datacenters. Fourth alone is a 8 core > Xeon box running VMWare ESX server with Windows 2003 running the fourth virtual > machine on that box. I spent $4500.00 for that server to -SUPPORT- cwop. I > also donate about 6 meg of tier1 quality backbone bandwidth every day to this > program. Unlike Findu, where Steve gets support from his users of that website, > the server sysops, who without there would be no network to use, we get zero > support from the user community. That' after thousands of dollars in hardware > and hundreds of hours of time. > > > > So how was I repaid for my generosity???? > > > > ? Requests to make changes to the client software went unanswered, > allowing things to spiral to where they are now. > > ? Every time anything happens to the network, the server sysops are > automatically crucified for doing something wrong. > > ? I can't think of the last time anyone stopped to tell me how much > they appreciate what I do. > > > > So it all boils down to this morning, when I posed the message at 10:24 am to > Seth about the standing of the software problems. > > > > Shortly afterward, I go this from Dave Helms: > > > > > > Dave, > > > > I, too, have absolutely no interest in revisiting the events of January-March > 2007 (publicly or otherwise). I am willing to support the notion of segmenting > CW traffic to improve stability of APRS-IS, to include developing a plan for > implementing the cwop.aprs.net round robin server name. In last year's crisis > mode, I agreed to contact virtually all CW and Ham weather station operators to > facilitate the switch from Tier2 servers to Core servers included on the > rotate.aprs.net RR server domain. In the future, I will rely on encouraging > individuals to read the CWOP News Page to advertise the RR server name change, > and through routine client application upgrades rather than making individual > email contacts. > > > > As we move forward, I request one thing... that we not make the "CW' (non-Ham) > volunteers second class users. Their contributions are equally valuable to NOAA > as validated Ham Radio users who send weather reports to APRS-IS and NOAA. If > weather traffic must be (temporarily) throttled back to preserve APRS-IS > stability, then all weather contributions should be equally reduced, Ham and > non-Hams alike. > > > > Going forward, we need to find a way to add capacity to the APRS-IS to take > the load of the expected growing number of users. Hence, we need to develop an > architecture that will allow exploitation of both Ham Radio APRS-IS server > resources and non-Ham servers which may be available. These non-Ham servers and > bandwidth may be donated from government, universities, and private sector > groups. > > > > Regards, > > > > Dave > > > > > > > > > > Finally after a year, Dave concedes to allowing us to start using > CWOP.APRS.NET. Well the problem is that it's a day late, and a dollar short. > He further more says it'll only be a news post on a website, and for when folks > update software. Well if the last change to rotate is any indication of how > well that will go, there still are 800 of you using the old servers to this DAY. > So if he's not going to help promote it, we'll have a network that is in > shambles here in the next 90 days due to load. I just don't think Dave > understands the gravity of the problem. > > > > Then the straw that broke the camel's back was in the second paragraph. > Insisting that ham stations also be reduced to that of non hams. Well first > off, ham stations are generally sharing a very limited amount of bandwidth on > 144.390mhz (1200bps). From peer pressure alone, no ham will beacon weather more > than every 5 minutes without hearing about it from other local hams. We have a > system in place that works well, using nothing but peer pressure. > > > > The fact that he's insisting hams back off, on their own network, that was > built by hams, really offended me, and so far most of the hams' I've shared > this with seem to agree. Dave and the NWS pay NOTHING for what we do on the > network. How dare he tell us what to do on our own network. CWOP are GUESTS on > our network. Dave has in the past shown callous towards the hams, never this > blatant, but I've seen it in other messages. This disgusts me. This would be > no different than me coming into your house as a guest and telling you can only > spend 5 hours a day watching TV. > > > > So I write Dave back a -very- long message, and at this point I'm very upset > about this. I lay blame for the problems we have right now where it needs to be > placed, squarely on his shoulders. He was the reason the white paper was given > up on, as if it wasn't Dave's way, it was no good. Yeah, I'll admit my message > to him was long, and rather blunt. I was direct with him, and factually telling > him it's come to a point where change has to occur, or the network will change > around him. > > > > Well, here was his response to that: > > > > > > > > > > > > Dave, > > > > In the time your spent wiring this email, you would could have instead been > making progress on the a white paper your said you would complete last year. > Don't blame the client developers for something you knew that was needed over a > year ago. > > > > Dave Helms > > > > > > > > > > Accusing me of not finishing a document that he opposed?!!?!?!?!?!! Excuse > me? Draft 3 was what would have gotten published had he not objected to the > hostname. The rest -is- done right now. > > > > Well if the NWS employee that is supposed to be our liaison to the developers > is taking this attitude towards me, with the thousands I have spent to support > CWOP, I hereby pull my official support from the program. > > > > I realize this will be a heavy hit for CWOP, as I take a full one half of the > server capacity with me. I am greatly sorry for that. The last people I > wanted to hurt were the ones I was out to help, but with Dave Helms at the > "Helm", it's over for me. I cannot continue to be badgered, beaten, ignored, > then finally blamed for it to begin with over a volunteer operation. > > > > If any of you would like to talk about this more, I encourage you to send me a > private e-mail at dave at aprsfl.net. I'm sure this message will get me banned on > this forum, so I'll not be checking back. > > > > I'm glad I've been able to help up till now, and I really am truly sorry for > going out this way, but I just can't go on with this type of pervasive > attitude.. Let alone one that is so callous towards the folks that this network > was built for, ham radio operators. > > > > R