From steve at softwx.com Tue Jan 1 15:54:48 2008 From: steve at softwx.com (Steve) Date: Tue, 1 Jan 2008 14:54:48 -0700 Subject: [wxqc] And another field goes bad In-Reply-To: References: Message-ID: <27E3781644354FF880E1AF835AD4B677@shrek> That's a good idea. I think I will add that to the next version of VPLive. Steve ----- Original Message ----- From: "Keith Miller" To: "'Discussion of weather data quality issues'" Sent: 12/31/2007 8:26 PM Subject: Re: [wxqc] And another field goes bad > > That's a pet peeve of mine, all software should allow for the > data of know bad sensor(s) to not be sent. Most all seem to > expect everything to be perfect and don't even make an effort > to check for obvious problems. > > > Keith > -- > CW5250 => http://weather.stadhaugh.com > http://weather.gladstonefamily.net/site/C5250 > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From Karl.Uppiano at verizon.net Tue Jan 1 16:10:58 2008 From: Karl.Uppiano at verizon.net (Karl Uppiano) Date: Tue, 01 Jan 2008 14:10:58 -0800 Subject: [wxqc] And another field goes bad References: <27E3781644354FF880E1AF835AD4B677@shrek> Message-ID: <74944F7BB0D04D15BCC4873993077DE2@Inspiron> I think a manual override would be more practical than attempts in software to recognize sensor problems. I think you would have problems distinguishing between false positives and simply unusual weather patterns. The only reliable way that I can think of to automatically switch off a sensor would be to have redundant sensors, and detect out-of-tolerance differences between the sensors. Regards, Karl Uppiano http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 http://mysite.verizon.net/Karl_Uppiano/weather.html -------------------------------------------------- From: "Steve" Sent: Tuesday, January 01, 2008 1:54 PM To: "Discussion of weather data quality issues" Subject: Re: [wxqc] And another field goes bad > That's a good idea. I think I will add that to the next version of VPLive. > > Steve > > ----- Original Message ----- > From: "Keith Miller" > To: "'Discussion of weather data quality issues'" > > Sent: 12/31/2007 8:26 PM > Subject: Re: [wxqc] And another field goes bad > > >> >> That's a pet peeve of mine, all software should allow for the >> data of know bad sensor(s) to not be sent. Most all seem to >> expect everything to be perfect and don't even make an effort >> to check for obvious problems. >> >> >> Keith >> -- >> CW5250 => http://weather.stadhaugh.com >> http://weather.gladstonefamily.net/site/C5250 >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From kc9dnq at comcast.net Wed Jan 2 10:06:56 2008 From: kc9dnq at comcast.net (Bill) Date: Wed, 02 Jan 2008 10:06:56 -0600 Subject: [wxqc] And another field goes bad In-Reply-To: <74944F7BB0D04D15BCC4873993077DE2@Inspiron> References: <27E3781644354FF880E1AF835AD4B677@shrek> <74944F7BB0D04D15BCC4873993077DE2@Inspiron> Message-ID: <477BB6A0.5000507@comcast.net> Good discussion. I too would prefer that the software would allow me to determine which wx parameters I wish to be included in the cwop data string. Not sure if this is what Steve or Keith were thinking, by my ideal scenario would include toggle switches/check boxes in the cwop setup GUI to *manually* turn a particular parameter on or off. For example if my wind sensor went out, instead of sending bad wind data, I could chose to send no wind data at all until such time that the sensor was repaired or replaced. This would be ideal for others that may not want to transmit a particular value perhaps because for whatever reasons they are unable to site a particular sensor properly. Bill www.gooselakeweather.com Karl Uppiano wrote: > I think a manual override would be more practical than attempts in software > to recognize sensor problems. I think you would have problems distinguishing > between false positives and simply unusual weather patterns. The only > reliable way that I can think of to automatically switch off a sensor would > be to have redundant sensors, and detect out-of-tolerance differences > between the sensors. > > Regards, > Karl Uppiano > > http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 > http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 > http://mysite.verizon.net/Karl_Uppiano/weather.html > > -------------------------------------------------- > From: "Steve" > Sent: Tuesday, January 01, 2008 1:54 PM > To: "Discussion of weather data quality issues" > > Subject: Re: [wxqc] And another field goes bad > > >> That's a good idea. I think I will add that to the next version of VPLive. >> >> Steve >> >> ----- Original Message ----- >> From: "Keith Miller" >> To: "'Discussion of weather data quality issues'" >> >> Sent: 12/31/2007 8:26 PM >> Subject: Re: [wxqc] And another field goes bad >> >> >> >>> That's a pet peeve of mine, all software should allow for the >>> data of know bad sensor(s) to not be sent. Most all seem to >>> expect everything to be perfect and don't even make an effort >>> to check for obvious problems. >>> >>> >>> Keith >>> -- >>> CW5250 => http://weather.stadhaugh.com >>> http://weather.gladstonefamily.net/site/C5250 >>> >>> _______________________________________________ >>> wxqc mailing list >>> Post messages to wxqc at lists.gladstonefamily.net >>> To unsubcribe or change delivery options, please go to: >>> http://server.gladstonefamily.net/mailman/listinfo/wxqc >>> >>> The contents of this message are the responsibility of the author. >>> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080102/abbd1fba/attachment.html From steve at softwx.com Wed Jan 2 10:52:09 2008 From: steve at softwx.com (Steve) Date: Wed, 2 Jan 2008 09:52:09 -0700 Subject: [wxqc] And another field goes bad In-Reply-To: <477BB6A0.5000507@comcast.net> References: <27E3781644354FF880E1AF835AD4B677@shrek><74944F7BB0D04D15BCC4873993077DE2@Inspiron> <477BB6A0.5000507@comcast.net> Message-ID: <479F175D2C9F4F12ADDAFE562C4A78DF@shrek> That's exactly what I had in mind: manual control on the CWOP setup screen, with checkboxes for each data item. This is not a quality check thing. It's just a way to shut off data submissions from a sensor the user has identified as not functioning properly or not present. Steve SoftWx ----- Original Message ----- From: Bill To: Discussion of weather data quality issues Sent: 01/02/2008 9:06 AM Subject: Re: [wxqc] And another field goes bad Good discussion. I too would prefer that the software would allow me to determine which wx parameters I wish to be included in the cwop data string. Not sure if this is what Steve or Keith were thinking, by my ideal scenario would include toggle switches/check boxes in the cwop setup GUI to *manually* turn a particular parameter on or off. For example if my wind sensor went out, instead of sending bad wind data, I could chose to send no wind data at all until such time that the sensor was repaired or replaced. This would be ideal for others that may not want to transmit a particular value perhaps because for whatever reasons they are unable to site a particular sensor properly. Bill www.gooselakeweather.com Karl Uppiano wrote: I think a manual override would be more practical than attempts in software to recognize sensor problems. I think you would have problems distinguishing between false positives and simply unusual weather patterns. The only reliable way that I can think of to automatically switch off a sensor would be to have redundant sensors, and detect out-of-tolerance differences between the sensors. Regards, Karl Uppiano http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 http://mysite.verizon.net/Karl_Uppiano/weather.html -------------------------------------------------- From: "Steve" Sent: Tuesday, January 01, 2008 1:54 PM To: "Discussion of weather data quality issues" Subject: Re: [wxqc] And another field goes bad That's a good idea. I think I will add that to the next version of VPLive. Steve ----- Original Message ----- From: "Keith Miller" To: "'Discussion of weather data quality issues'" Sent: 12/31/2007 8:26 PM Subject: Re: [wxqc] And another field goes bad That's a pet peeve of mine, all software should allow for the data of know bad sensor(s) to not be sent. Most all seem to expect everything to be perfect and don't even make an effort to check for obvious problems. Keith -- CW5250 => http://weather.stadhaugh.com http://weather.gladstonefamily.net/site/C5250 _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. ------------------------------------------------------------------------------ _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080102/8b795eb9/attachment-0001.html From brillig at gmail.com Wed Jan 2 11:05:14 2008 From: brillig at gmail.com (Victor Engel) Date: Wed, 2 Jan 2008 11:05:14 -0600 Subject: [wxqc] And another field goes bad In-Reply-To: <479F175D2C9F4F12ADDAFE562C4A78DF@shrek> References: <27E3781644354FF880E1AF835AD4B677@shrek> <74944F7BB0D04D15BCC4873993077DE2@Inspiron> <477BB6A0.5000507@comcast.net> <479F175D2C9F4F12ADDAFE562C4A78DF@shrek> Message-ID: Does the record format allow for non-entries? On Jan 2, 2008 10:52 AM, Steve wrote: > > > That's exactly what I had in mind: manual control on the CWOP setup screen, > with checkboxes for each data item. This is not a quality check thing. It's > just a way to shut off data submissions from a sensor the user has > identified as not functioning properly or not present. > > Steve > SoftWx > > ----- Original Message ----- > From: Bill > To: Discussion of weather data quality issues > > > Sent: 01/02/2008 9:06 AM > Subject: Re: [wxqc] And another field goes bad > > Good discussion. > I too would prefer that the software would allow me to determine which wx > parameters I wish to be included in the cwop data string. Not sure if this > is what Steve or Keith were thinking, by my ideal scenario would include > toggle switches/check boxes in the cwop setup GUI to *manually* turn a > particular parameter on or off. For example if my wind sensor went out, > instead of sending bad wind data, I could chose to send no wind data at all > until such time that the sensor was repaired or replaced. This would be > ideal for others that may not want to transmit a particular value perhaps > because for whatever reasons they are unable to site a particular sensor > properly. > > Bill > www.gooselakeweather.com > > > Karl Uppiano wrote: > I think a manual override would be more practical than attempts in software > to recognize sensor problems. I think you would have problems distinguishing > between false positives and simply unusual weather patterns. The only > reliable way that I can think of to automatically switch off a sensor would > be to have redundant sensors, and detect out-of-tolerance differences > between the sensors. > > Regards, > Karl Uppiano > > http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 > http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 > http://mysite.verizon.net/Karl_Uppiano/weather.html > > -------------------------------------------------- > From: "Steve" > Sent: Tuesday, January 01, 2008 1:54 PM > To: "Discussion of weather data quality issues" > > Subject: Re: [wxqc] And another field goes bad > > > > That's a good idea. I think I will add that to the next version of VPLive. > > Steve > > ----- Original Message ----- > From: "Keith Miller" > To: "'Discussion of weather data quality issues'" > > Sent: 12/31/2007 8:26 PM > Subject: Re: [wxqc] And another field goes bad > > > > > That's a pet peeve of mine, all software should allow for the > data of know bad sensor(s) to not be sent. Most all seem to > expect everything to be perfect and don't even make an effort > to check for obvious problems. > > > Keith > -- > CW5250 => http://weather.stadhaugh.com > http://weather.gladstonefamily.net/site/C5250 > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > > > > > ________________________________ > > > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. > From Karl.Uppiano at verizon.net Wed Jan 2 21:22:04 2008 From: Karl.Uppiano at verizon.net (Karl Uppiano) Date: Wed, 02 Jan 2008 19:22:04 -0800 Subject: [wxqc] And another field goes bad References: <27E3781644354FF880E1AF835AD4B677@shrek> <74944F7BB0D04D15BCC4873993077DE2@Inspiron> <477BB6A0.5000507@comcast.net> <479F175D2C9F4F12ADDAFE562C4A78DF@shrek> Message-ID: Yes, according to this, http://weather.gladstonefamily.net/aprswxnet.html, CWOP allows some sensor data to be omitted. -- Karl -------------------------------------------------- From: "Victor Engel" Sent: Wednesday, January 02, 2008 9:05 AM To: "Steve" ; "Discussion of weather data quality issues" Subject: Re: [wxqc] And another field goes bad > Does the record format allow for non-entries? > > On Jan 2, 2008 10:52 AM, Steve wrote: >> >> >> That's exactly what I had in mind: manual control on the CWOP setup >> screen, >> with checkboxes for each data item. This is not a quality check thing. >> It's >> just a way to shut off data submissions from a sensor the user has >> identified as not functioning properly or not present. >> >> Steve >> SoftWx >> >> ----- Original Message ----- >> From: Bill >> To: Discussion of weather data quality issues >> >> >> Sent: 01/02/2008 9:06 AM >> Subject: Re: [wxqc] And another field goes bad >> >> Good discussion. >> I too would prefer that the software would allow me to determine which wx >> parameters I wish to be included in the cwop data string. Not sure if >> this >> is what Steve or Keith were thinking, by my ideal scenario would include >> toggle switches/check boxes in the cwop setup GUI to *manually* turn a >> particular parameter on or off. For example if my wind sensor went out, >> instead of sending bad wind data, I could chose to send no wind data at >> all >> until such time that the sensor was repaired or replaced. This would be >> ideal for others that may not want to transmit a particular value perhaps >> because for whatever reasons they are unable to site a particular sensor >> properly. >> >> Bill >> www.gooselakeweather.com >> >> >> Karl Uppiano wrote: >> I think a manual override would be more practical than attempts in >> software >> to recognize sensor problems. I think you would have problems >> distinguishing >> between false positives and simply unusual weather patterns. The only >> reliable way that I can think of to automatically switch off a sensor >> would >> be to have redundant sensors, and detect out-of-tolerance differences >> between the sensors. >> >> Regards, >> Karl Uppiano >> >> http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 >> http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 >> http://mysite.verizon.net/Karl_Uppiano/weather.html >> >> -------------------------------------------------- >> From: "Steve" >> Sent: Tuesday, January 01, 2008 1:54 PM >> To: "Discussion of weather data quality issues" >> >> Subject: Re: [wxqc] And another field goes bad >> >> >> >> That's a good idea. I think I will add that to the next version of >> VPLive. >> >> Steve >> >> ----- Original Message ----- >> From: "Keith Miller" >> To: "'Discussion of weather data quality issues'" >> >> Sent: 12/31/2007 8:26 PM >> Subject: Re: [wxqc] And another field goes bad >> >> >> >> >> That's a pet peeve of mine, all software should allow for the >> data of know bad sensor(s) to not be sent. Most all seem to >> expect everything to be perfect and don't even make an effort >> to check for obvious problems. >> >> >> Keith >> -- >> CW5250 => http://weather.stadhaugh.com >> http://weather.gladstonefamily.net/site/C5250 >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> >> >> >> >> ________________________________ >> >> >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. >> > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From kdmiller at oldsgmail.com Wed Jan 2 22:05:21 2008 From: kdmiller at oldsgmail.com (Keith Miller) Date: Wed, 02 Jan 2008 23:05:21 -0500 Subject: [wxqc] And another field goes bad In-Reply-To: <74944F7BB0D04D15BCC4873993077DE2@Inspiron> Message-ID: <3EC102028DB04E37B2EFAFBD0EEC9268@sauron> > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of > Karl Uppiano > Sent: Tuesday, January 01, 2008 5:11 PM > To: Steve; Discussion of weather data quality issues > Subject: Re: [wxqc] And another field goes bad > > > I think a manual override would be more practical than That should be a minimum. > attempts in software to recognize sensor problems. I think > you would have problems distinguishing between false > positives and simply unusual weather patterns. The only > reliable way that I can think of to automatically switch off > a sensor would be to have redundant sensors, and detect > out-of-tolerance differences between the sensors. > It would be more difficult, but basic checks could be done. For example, a Davis VP ISS temperature is reported in 2 bytes with 1/10 resolution (I assume signed), I'd have to pull out my notes to be certain, but it seems to me it's sent in hex. Which would allow for a range of -409.5 to 409.5 deg F, obviously one could flag anything not within, say, -150 to 150F as invalid. If one really wanted to get fancy, it wouldn't be too hard to check for a stuck sensor. Check new value against last value, if same incriment a counter. If the same value persisted for maybe 24 hours, I would think it would be suspect. This is really just basic error checking, which many programmers overlook. Keith -- CW5250 => http://weather.stadhaugh.com http://weather.gladstonefamily.net/site/C5250 From steve at softwx.com Wed Jan 2 23:35:39 2008 From: steve at softwx.com (Steve) Date: Wed, 2 Jan 2008 22:35:39 -0700 Subject: [wxqc] And another field goes bad In-Reply-To: <3EC102028DB04E37B2EFAFBD0EEC9268@sauron> References: <3EC102028DB04E37B2EFAFBD0EEC9268@sauron> Message-ID: <3021F1756F804E59B97121B8ADC89639@shrek> I can't speak for the other software programs, but I know that VPLive already has minimal sanity checking. The various data items that are sent APRS/CWOP are not instanteous readings, but are processed a bit, depending on the data item. Some items such as humidity, temp and wind are averaged over specified periods of time. Others, such as wind gust and barometer report the highest or lowest value over a specified time period. VPLive has code that acts as accumulators for maintaining/calculating these running values. Each time raw data is received from the console, each data element is sanity checked against an acceptable range of data values before being pushed into the accumulators. This weeds out clearly bogus data from affecting the running values sent to APRS/CWOP. For instance, if a sensor bellies up or has a temporary glitch, the data received will often be hex FF (all bits set). This is probably why stations with problems often send very high values to CWOP. When the APRS data packet is constructed, if the accumulator for a data item had no valid data points, then that data item is omitted from the packet, or, if it's a mandatory field, the field is filled with periods (...) to denote missing data per the specs. The ASOS User Guide (aum-toc.pdf) is a good source for seeing how serious equipment processes the sensor data. I found it very helpful. Detecting a stuck sensor is a little trickier. I would not be inclined to have the software automatically block data that looked stuck, but I could see value in providing an alert to the user when a suspected condition like this is detected. Then the user could use the manual, per sensor control of what gets sent as they saw fit. Steve SoftWx ----- Original Message ----- From: "Keith Miller" To: "'Discussion of weather data quality issues'" Sent: 01/02/2008 9:05 PM Subject: Re: [wxqc] And another field goes bad >> -----Original Message----- >> From: wxqc-bounces at lists.gladstonefamily.net >> [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of >> Karl Uppiano >> Sent: Tuesday, January 01, 2008 5:11 PM >> To: Steve; Discussion of weather data quality issues >> Subject: Re: [wxqc] And another field goes bad >> >> >> I think a manual override would be more practical than > > That should be a minimum. > >> attempts in software to recognize sensor problems. I think >> you would have problems distinguishing between false >> positives and simply unusual weather patterns. The only >> reliable way that I can think of to automatically switch off >> a sensor would be to have redundant sensors, and detect >> out-of-tolerance differences between the sensors. >> > > It would be more difficult, but basic checks could be done. > For example, a Davis VP ISS temperature is reported in 2 > bytes with 1/10 resolution (I assume signed), I'd have to > pull out my notes to be certain, but it seems to me it's > sent in hex. Which would allow for a range of -409.5 to > 409.5 deg F, obviously one could flag anything not within, > say, -150 to 150F as invalid. > > If one really wanted to get fancy, it wouldn't be too hard > to check for a stuck sensor. Check new value against last > value, if same incriment a counter. If the same value > persisted for maybe 24 hours, I would think it would be > suspect. > > This is really just basic error checking, which many > programmers overlook. > > > Keith > -- > CW5250 => http://weather.stadhaugh.com > http://weather.gladstonefamily.net/site/C5250 > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From kdmiller at oldsgmail.com Thu Jan 3 00:09:45 2008 From: kdmiller at oldsgmail.com (Keith Miller) Date: Thu, 03 Jan 2008 01:09:45 -0500 Subject: [wxqc] And another field goes bad In-Reply-To: <3021F1756F804E59B97121B8ADC89639@shrek> Message-ID: <719A09D155924B30B210D83FBE99C770@sauron> > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of Steve > Sent: Thursday, January 03, 2008 12:36 AM > To: Discussion of weather data quality issues > Subject: Re: [wxqc] And another field goes bad > > > This weeds out clearly bogus data from affecting the running > values sent to APRS/CWOP. For instance, if a sensor bellies up > or has a temporary glitch, the data received will often be hex > FF (all bits set). Something many seem to ignore... Ahh, you keep tempting me, but I really, really do much like the LCD capability of WxSolution, ie: http://www.stadhaugh.com/flora/lcd.php?lcd=200712 not to mention the Heathkit like user interface, and the like remote client. > Detecting a stuck sensor is a little trickier. I would not be > inclined to have the software automatically block data that > looked stuck, but I could see value in providing an alert > to the user when a suspected condition like this is detected. > That was my thought, I kinda doubt anything would remain the same for a period as long as 24 hours, but it might be possible. So just flag it as a possible issue. Keith -- CW5250 => http://weather.stadhaugh.com http://weather.gladstonefamily.net/site/C5250 From eholmes at aeneas.net Thu Jan 3 21:42:11 2008 From: eholmes at aeneas.net (Eddie Holmes) Date: Thu, 3 Jan 2008 21:42:11 -0600 Subject: [wxqc] Sensor Deviation at Low Dew Points Message-ID: <000c01c84e83$cc34bdf0$0401a8c0@EDDIESPC> My dew point sensor tracks well with the analysis throughout the course of a year EXCEPT when we have arctic air intrusion and dew point temps are in the single digits. My sensor seems to lag behind in recovering with the DP's when it starts rising again. Note my data quality report (Dew Point Panel): http://pond1.gladstonefamily.net:8080/cgi-bin/wxqchart.pl?site=C4199 I've seen this for two seasons running. I operate a Davis WMII in a Cotton Region instrument shelter (unaspirated). Comments? - Eddie -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080103/383c6b28/attachment-0001.html From Karl.Uppiano at verizon.net Thu Jan 3 21:45:43 2008 From: Karl.Uppiano at verizon.net (Karl Uppiano) Date: Thu, 03 Jan 2008 19:45:43 -0800 Subject: [wxqc] And another field goes bad References: <3EC102028DB04E37B2EFAFBD0EEC9268@sauron> <3021F1756F804E59B97121B8ADC89639@shrek> Message-ID: <2866AC7AFC024108B9C3218D1039899A@Inspiron> WxService does things like this as well -- averaging data, looking for obviously out of range data, and verifying checksums. However "stuck" sensors don't necessarily keep sending exactly the same codes at each sample. For example, a temperature-compensated humidity sensor or pressure sensor might still change a bit with temperature in spite of being stuck. It gets more interesting then, because "stuck" values might have to fall within some difficult-to-predict tolerance window that still might be legitimate weather data. Pattern recognition that is as good as we humans have developed is still very expensive if not impossible to do in software. I can look at a chart and see immediately that a device is stuck, but I can't design a simple, affordable software algorithm with the same high quality. Regards, Karl Uppiano http://www.findu.com/cgi-bin/wxpage.cgi?call=CW3375 http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KWAFERND7 http://mysite.verizon.net/Karl_Uppiano/weather.html -------------------------------------------------- From: "Steve" Sent: Wednesday, January 02, 2008 9:35 PM To: "Discussion of weather data quality issues" Subject: Re: [wxqc] And another field goes bad > I can't speak for the other software programs, but I know that VPLive > already has minimal sanity checking. The various data items that are sent > APRS/CWOP are not instanteous readings, but are processed a bit, depending > on the data item. Some items such as humidity, temp and wind are averaged > over specified periods of time. Others, such as wind gust and barometer > report the highest or lowest value over a specified time period. VPLive > has > code that acts as accumulators for maintaining/calculating these running > values. Each time raw data is received from the console, each data element > is sanity checked against an acceptable range of data values before being > pushed into the accumulators. This weeds out clearly bogus data from > affecting the running values sent to APRS/CWOP. For instance, if a sensor > bellies up or has a temporary glitch, the data received will often be hex > FF > (all bits set). This is probably why stations with problems often send > very > high values to CWOP. When the APRS data packet is constructed, if the > accumulator for a data item had no valid data points, then that data item > is > omitted from the packet, or, if it's a mandatory field, the field is > filled > with periods (...) to denote missing data per the specs. > > The ASOS User Guide (aum-toc.pdf) is a good source for seeing how serious > equipment processes the sensor data. I found it very helpful. > > Detecting a stuck sensor is a little trickier. I would not be inclined to > have the software automatically block data that looked stuck, but I could > see value in providing an alert to the user when a suspected condition > like > this is detected. Then the user could use the manual, per sensor control > of > what gets sent as they saw fit. > > Steve > SoftWx > > ----- Original Message ----- > From: "Keith Miller" > To: "'Discussion of weather data quality issues'" > > Sent: 01/02/2008 9:05 PM > Subject: Re: [wxqc] And another field goes bad > > >>> -----Original Message----- >>> From: wxqc-bounces at lists.gladstonefamily.net >>> [mailto:wxqc-bounces at lists.gladstonefamily.net]On Behalf Of >>> Karl Uppiano >>> Sent: Tuesday, January 01, 2008 5:11 PM >>> To: Steve; Discussion of weather data quality issues >>> Subject: Re: [wxqc] And another field goes bad >>> >>> >>> I think a manual override would be more practical than >> >> That should be a minimum. >> >>> attempts in software to recognize sensor problems. I think >>> you would have problems distinguishing between false >>> positives and simply unusual weather patterns. The only >>> reliable way that I can think of to automatically switch off >>> a sensor would be to have redundant sensors, and detect >>> out-of-tolerance differences between the sensors. >>> >> >> It would be more difficult, but basic checks could be done. >> For example, a Davis VP ISS temperature is reported in 2 >> bytes with 1/10 resolution (I assume signed), I'd have to >> pull out my notes to be certain, but it seems to me it's >> sent in hex. Which would allow for a range of -409.5 to >> 409.5 deg F, obviously one could flag anything not within, >> say, -150 to 150F as invalid. >> >> If one really wanted to get fancy, it wouldn't be too hard >> to check for a stuck sensor. Check new value against last >> value, if same incriment a counter. If the same value >> persisted for maybe 24 hours, I would think it would be >> suspect. >> >> This is really just basic error checking, which many >> programmers overlook. >> >> >> Keith >> -- >> CW5250 => http://weather.stadhaugh.com >> http://weather.gladstonefamily.net/site/C5250 >> >> _______________________________________________ >> wxqc mailing list >> Post messages to wxqc at lists.gladstonefamily.net >> To unsubcribe or change delivery options, please go to: >> http://server.gladstonefamily.net/mailman/listinfo/wxqc >> >> The contents of this message are the responsibility of the author. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From steve at dimse.com Sun Jan 6 14:57:26 2008 From: steve at dimse.com (Steve Dimse) Date: Sun, 6 Jan 2008 15:57:26 -0500 Subject: [wxqc] 2007 Stats In-Reply-To: References: <27E3781644354FF880E1AF835AD4B677@shrek> <74944F7BB0D04D15BCC4873993077DE2@Inspiron> <477BB6A0.5000507@comcast.net> <479F175D2C9F4F12ADDAFE562C4A78DF@shrek> Message-ID: <73141944-A569-4A1A-8175-890EFBF4BDC9@dimse.com> I'm doing the end-of-year table maintenance on findU, and thought I's share some stats. In 2007 findU archived 422,262,687 weather reports, an increase of 29.5% over the 320M last year. In all, the findU archive contains over 1.24 billion reports going back to March, 2000. I remember thinking the 12 million logged in 2000 were a lot... Steve From lwood at mountainbase.com Sun Jan 6 16:19:14 2008 From: lwood at mountainbase.com (lwood at mountainbase.com) Date: Sun, 06 Jan 2008 15:19:14 -0700 Subject: [wxqc] DSL Connection down Message-ID: <20080106151914.b014af6efb42cca08111bb1cc252e7a0.d8225498e2.wbe@email.secureserver.net> An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080106/9f77303b/attachment.html From john at strandediniowa.com Sun Jan 6 18:02:19 2008 From: john at strandediniowa.com (John) Date: Mon, 7 Jan 2008 0:02:19 +0000 Subject: [wxqc] Bad yearly high/low data? Message-ID: <6d51a6c7f524bfed33ee601915844d6f@mail.e-rice.net> I'm using a Davis VP2 Plus with Virtual Weather Station and a soil moisture/temperature module. When the VWS locks up or otherwise fails, the soil temp values will default to zero in the VWS database. The data in the console is fine. Even after manually editing the bad values in the database, the details.htm page shows 0.00 as the yearly low. Is there data stored somewhere else that must be updated, or can I force VWS to recalculate the yearly extremes? John CW8283 http://www.strandediniowa.com From seth at ieee.org Tue Jan 8 22:16:04 2008 From: seth at ieee.org (Seth Cohen) Date: Tue, 08 Jan 2008 23:16:04 -0500 Subject: [wxqc] Need Help With Siting In-Reply-To: <043101c8471a$cd6f3bc0$684db340$@org> References: <043101c8471a$cd6f3bc0$684db340$@org> Message-ID: <021001c85276$5a976930$0fc63b90$@org> I think I probably have a siting problem with my weather station. I have read the document at http://mywebpages.comcast.net/dshelms/CWOP_Guide.pdf but I think that I still may need some help with fine tuning. The night time temperatures I record are usually lower than the surrounding stations. Last night it was as much as 10 degrees colder. I have a second weather station located about 15 feet away that gave the same temperature readings, so I don't think I have a hardware problem with the temperature sensor. Should I be concerned? Do I need to relocate the temperature sensor? See the graph at: http://weather.gladstonefamily.net/cgi-bin/wxqchart.pl?site=as787 Based on the sharp rise and fall in temperature between 10:00 and 11:00 local time, I recognize that I also have a solar heating problem that I hope to address by adding fan aspiration to my solar shield. The temperature sensor is currently located about 5 feet off the ground in as open an area as I have in my back yard. It is over a grass lawn and is inside a solar shield with no fan aspiration. I also seem to have a problem where the humidity reads too high right after sunset and before sunrise. This has been fairly consistent, and is not always reflected in the second weather station. Is it possible that I'm getting some condensation when the temperature drops after sunset? Assuming that is the case, will adding fan aspiration help to address that as well? Should I consider relocating the humidity sensor, or has the sensor possibly gone bad? Thanks for any advice somebody may want to share, Seth Cohen KB3HHA/AS787 From drkruse at covepets.net Fri Jan 11 12:23:47 2008 From: drkruse at covepets.net (Kevin Kruse, DVM) Date: Fri, 11 Jan 2008 12:23:47 -0600 Subject: [wxqc] Calibrating Oregon Scientific WMR 968 Dewpoint Message-ID: Can anyone give me help in getting the Dewpoint calibrated on my weather station. Am using Oregon Scientific WMR 968 station and have Weathertracker as the software. The Temp/Humidity sensor is enclosed in a radiation shield to avoid abnormal reading due to sunlight. Thanks Kevin E. Kruse, DVM Hill Country Animal Hospital 103 Wolfe Rd. Copperas Cove, TX 76522 254-547-8881 drkruse at covepets.net -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080111/0334359a/attachment.html From cblack at rdhbe.com Fri Jan 11 20:39:53 2008 From: cblack at rdhbe.com (Christopher Black) Date: Fri, 11 Jan 2008 18:39:53 -0800 Subject: [wxqc] Help with Weather Underground Message-ID: <6E9F0C251E741948952E6385A93419A1AD429B@RDHVEXCH.rdhbe.com> Not sure if this is the right group to post this question too, but I need help with my Weather Station and Weather Underground. I have a WS 2308U weather station, I know not the best, but anyways. I am using Heavy Weather which is linked to Virtual Weather Station which is uploading to Weather Underground (Rapid Fire). Well Weather Underground is recording the right daily rain fall, but it is not updating it within my table of rainfall or my graph (i.e. hourly rain fall). Does anyone know why, and how I might fix it. Link to Weather Station: http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IBCVANC O8 Regards, Christopher -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080111/8704dbff/attachment.html From unkajim at comcast.net Fri Jan 11 23:27:18 2008 From: unkajim at comcast.net (Jim Pace) Date: Fri, 11 Jan 2008 21:27:18 -0800 Subject: [wxqc] [personal] Need Help With Siting References: <043101c8471a$cd6f3bc0$684db340$@org> <021001c85276$5a976930$0fc63b90$@org> Message-ID: <00bc01c854db$cd870c90$6500a8c0@jim913fn14t9kh> Seth, I don't think you have any siting issue at all. Your description of conditions mirror my site almost exactly. We're both situated in a microclimate area and although our readings are true, they are going to get flagged as being in error sometimes. It bugs me some as it probably does you but what are we to do? I have often had to scrape that 10 degree too low temperature "error" off my windshield in order to see to drive to work. Jim ----- Original Message ----- From: "Seth Cohen" To: "'Discussion of weather data quality issues'" Sent: Tuesday, January 08, 2008 8:16 PM Subject: [personal] [wxqc] Need Help With Siting >I think I probably have a siting problem with my weather station. I have > read the document at http://mywebpages.comcast.net/dshelms/CWOP_Guide.pdf > but I think that I still may need some help with fine tuning. > > The night time temperatures I record are usually lower than the > surrounding > stations. Last night it was as much as 10 degrees colder. I have a > second > weather station located about 15 feet away that gave the same temperature > readings, so I don't think I have a hardware problem with the temperature > sensor. Should I be concerned? Do I need to relocate the temperature > sensor? > > See the graph at: > > http://weather.gladstonefamily.net/cgi-bin/wxqchart.pl?site=as787 > > Based on the sharp rise and fall in temperature between 10:00 and 11:00 > local time, I recognize that I also have a solar heating problem that I > hope > to address by adding fan aspiration to my solar shield. The temperature > sensor is currently located about 5 feet off the ground in as open an area > as I have in my back yard. It is over a grass lawn and is inside a solar > shield with no fan aspiration. > > I also seem to have a problem where the humidity reads too high right > after > sunset and before sunrise. This has been fairly consistent, and is not > always reflected in the second weather station. Is it possible that I'm > getting some condensation when the temperature drops after sunset? > Assuming > that is the case, will adding fan aspiration help to address that as well? > Should I consider relocating the humidity sensor, or has the sensor > possibly > gone bad? > > Thanks for any advice somebody may want to share, > Seth Cohen > KB3HHA/AS787 > > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From bryce at azlab.net Fri Jan 11 23:35:47 2008 From: bryce at azlab.net (Bryce Alexander) Date: Fri, 11 Jan 2008 22:35:47 -0700 Subject: [wxqc] Need Help With Siting In-Reply-To: <021001c85276$5a976930$0fc63b90$@org> References: <043101c8471a$cd6f3bc0$684db340$@org> <021001c85276$5a976930$0fc63b90$@org> Message-ID: <0ff501c854dc$ff267d70$03371e0a@Artemis> Sometimes it isn't the siting that is at issue, there are micro-climates where the differences really exist. You mentioned that another temperature sensor within 15 feet is reading the same thing, what are the odds that both are wrong? The humidity seems a little off, you may want to see if you can get your hands on a sling psycrometer to determine if the readings of your sensor match up. I was able to borrow one from the facilities person where I work, he uses it to check the air conditioning. You can also find them pretty frequently on ebay if you want to purchase one. It is a kind of measure twice, cut once concept, verify that your instruments are accurate, then if it turns out that the information is correct, don't worry about how it stacks up to the other sensors around you. I would still aspirate the temp/humidity sensor, I found that the stagnate air inside some of the solar shields do have a negative impact on the accuracy of the humidity sensor. Getting back to the siting issues, what you say sounds like you are doing the right thing. Try verifying your readings with other instruments to determine if you are just in a microclimate that is naturally different from the sensors a few miles away or if aspirating will clear up the differences you are seeing. Good luck. -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Seth Cohen Sent: Tuesday, January 08, 2008 9:16 PM To: 'Discussion of weather data quality issues' Subject: [wxqc] Need Help With Siting I think I probably have a siting problem with my weather station. I have read the document at http://mywebpages.comcast.net/dshelms/CWOP_Guide.pdf but I think that I still may need some help with fine tuning. The night time temperatures I record are usually lower than the surrounding stations. Last night it was as much as 10 degrees colder. I have a second weather station located about 15 feet away that gave the same temperature readings, so I don't think I have a hardware problem with the temperature sensor. Should I be concerned? Do I need to relocate the temperature sensor? See the graph at: http://weather.gladstonefamily.net/cgi-bin/wxqchart.pl?site=as787 Based on the sharp rise and fall in temperature between 10:00 and 11:00 local time, I recognize that I also have a solar heating problem that I hope to address by adding fan aspiration to my solar shield. The temperature sensor is currently located about 5 feet off the ground in as open an area as I have in my back yard. It is over a grass lawn and is inside a solar shield with no fan aspiration. I also seem to have a problem where the humidity reads too high right after sunset and before sunrise. This has been fairly consistent, and is not always reflected in the second weather station. Is it possible that I'm getting some condensation when the temperature drops after sunset? Assuming that is the case, will adding fan aspiration help to address that as well? Should I consider relocating the humidity sensor, or has the sensor possibly gone bad? Thanks for any advice somebody may want to share, Seth Cohen KB3HHA/AS787 _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. From m.robbie at mchsi.com Sat Jan 12 12:52:21 2008 From: m.robbie at mchsi.com (Mark) Date: Sat, 12 Jan 2008 12:52:21 -0600 Subject: [wxqc] Need recommendation on QC adjustments Message-ID: <000001c8554c$43664370$aa00a8c0@Owner> Newby Question: I periodically will make slight adjustments to try keep my system in check but for the past month or so there seems to be a site near me that was possibly a bit off (KF4TTB-2). http://weather.gladstonefamily.net/site/C8100 I've been holding off on making adjustments but thinking of computing manually using the other local's near me, or possibly just waiting a bit longer as it looks as though the site is coming back in line. (I'm not sure if it is used in the MADIS QC computations and didn't want to make corrections if it was). Suggestions? Thanks Mark CW8100 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080112/364f8973/attachment.html From seth at ieee.org Sat Jan 12 16:38:06 2008 From: seth at ieee.org (Seth Cohen) Date: Sat, 12 Jan 2008 17:38:06 -0500 Subject: [wxqc] Need Help With Siting In-Reply-To: <0ff501c854dc$ff267d70$03371e0a@Artemis> References: <043101c8471a$cd6f3bc0$684db340$@org> <021001c85276$5a976930$0fc63b90$@org> <0ff501c854dc$ff267d70$03371e0a@Artemis> Message-ID: <03c301c8556b$cdc63600$6952a200$@org> Jim and Bryce, Thanks for confirming what I suspected regarding my temperature readings. I installed a fan in my solar shield today, so I'm anxious to see what effect that has on both my temp and humidity readings. I'll let it run for a week and see what happens. I'll also check about verifying the accuracy of my humidity sensor. It was affected by the weather sealant I used (electrician's putty), and I needed to remove all of it and use something else (petroleum grease). I haven't completely trusted it since then. Thanks again for your help, Seth CWOP AS787 -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Bryce Alexander Sent: Saturday, January 12, 2008 12:36 AM To: 'Discussion of weather data quality issues' Subject: Re: [wxqc] Need Help With Siting Sometimes it isn't the siting that is at issue, there are micro-climates where the differences really exist. You mentioned that another temperature sensor within 15 feet is reading the same thing, what are the odds that both are wrong? The humidity seems a little off, you may want to see if you can get your hands on a sling psycrometer to determine if the readings of your sensor match up. I was able to borrow one from the facilities person where I work, he uses it to check the air conditioning. You can also find them pretty frequently on ebay if you want to purchase one. It is a kind of measure twice, cut once concept, verify that your instruments are accurate, then if it turns out that the information is correct, don't worry about how it stacks up to the other sensors around you. I would still aspirate the temp/humidity sensor, I found that the stagnate air inside some of the solar shields do have a negative impact on the accuracy of the humidity sensor. Getting back to the siting issues, what you say sounds like you are doing the right thing. Try verifying your readings with other instruments to determine if you are just in a microclimate that is naturally different from the sensors a few miles away or if aspirating will clear up the differences you are seeing. Good luck. -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Seth Cohen Sent: Tuesday, January 08, 2008 9:16 PM To: 'Discussion of weather data quality issues' Subject: [wxqc] Need Help With Siting I think I probably have a siting problem with my weather station. I have read the document at http://mywebpages.comcast.net/dshelms/CWOP_Guide.pdf but I think that I still may need some help with fine tuning. The night time temperatures I record are usually lower than the surrounding stations. Last night it was as much as 10 degrees colder. I have a second weather station located about 15 feet away that gave the same temperature readings, so I don't think I have a hardware problem with the temperature sensor. Should I be concerned? Do I need to relocate the temperature sensor? See the graph at: http://weather.gladstonefamily.net/cgi-bin/wxqchart.pl?site=as787 Based on the sharp rise and fall in temperature between 10:00 and 11:00 local time, I recognize that I also have a solar heating problem that I hope to address by adding fan aspiration to my solar shield. The temperature sensor is currently located about 5 feet off the ground in as open an area as I have in my back yard. It is over a grass lawn and is inside a solar shield with no fan aspiration. I also seem to have a problem where the humidity reads too high right after sunset and before sunrise. This has been fairly consistent, and is not always reflected in the second weather station. Is it possible that I'm getting some condensation when the temperature drops after sunset? Assuming that is the case, will adding fan aspiration help to address that as well? Should I consider relocating the humidity sensor, or has the sensor possibly gone bad? Thanks for any advice somebody may want to share, Seth Cohen KB3HHA/AS787 _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. _______________________________________________ wxqc mailing list Post messages to wxqc at lists.gladstonefamily.net To unsubcribe or change delivery options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of this message are the responsibility of the author. -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 5104 bytes Desc: not available Url : http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080112/adf0dbe2/attachment.bin From kwalrath at accubak.net Sat Jan 12 18:22:04 2008 From: kwalrath at accubak.net (kwalrath) Date: Sat, 12 Jan 2008 18:22:04 -0600 Subject: [wxqc] Cw4856 failure to communicate Message-ID: <000001c8557a$54023120$fc069360$@net> The CW4856 site was down due to the winds turning my WI FI antenna during the recent storms. I have that fixed and have discovered a couple more problems. My wind speed seems to quit during excessive winds or it may be due to the moisture. The rain monitor also quits during hi winds and rain, I will fix these problems in the near future, all the other sensors seem to be accurate. ken -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080112/e3e28647/attachment.html From cmowen at att.net Mon Jan 14 07:31:06 2008 From: cmowen at att.net (cmowen at att.net) Date: Mon, 14 Jan 2008 13:31:06 +0000 Subject: [wxqc] Some Stations down through APRS/Findu Message-ID: <011420081331.1922.478B6419000F0AE00000078222230704929B0A02D29B9B0EBF020A9901030C@att.net> My station and others nearby have stopped getting to the findu site. But I see that several are still working so the whole system is not down. My Raw data suggests that I was sending to "Third" predomately right before the fail. Others that are down show alot of flip-flopping. Even the ones that are up seem to be going between "First" and "Fourth" alot. Any Ideas where the breakdown is? Data getting to APRS? MADIS? just not displaying? Charles CW4870 -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/10a9bc6a/attachment-0001.html From steve at dimse.com Mon Jan 14 10:24:57 2008 From: steve at dimse.com (Steve Dimse) Date: Mon, 14 Jan 2008 11:24:57 -0500 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420081331.1922.478B6419000F0AE00000078222230704929B0A02D29B9B0EBF020A9901030C@att.net> References: <011420081331.1922.478B6419000F0AE00000078222230704929B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: On Jan 14, 2008, at 8:31 AM, cmowen at att.net wrote: > My station and others nearby have stopped getting to the findu site. > But I see that several are still working so the whole system is not > down. My Raw data suggests that I was sending to "Third" predomately > right before the fail. Others that are down show alot of flip- > flopping. Even the ones that are up seem to be going between "First" > and "Fourth" alot. > > Any Ideas where the breakdown is? Data getting to APRS? MADIS? just > not displaying? Ideally each stations should be flip-flopping. When your weather software asks to connect to rotate.aprs.net, it is given a list of the four collection hubs, in what is essentially a random order. Your system is supposed to use the list in this order and not cache it. Windows is notorious for not following the rules though, so there is often unwanted caching of the data. I can't say specifically why you and your friends are having problems, only that the system is up and functioning normally, with a normal number of users sending data, there has been no drop in output numbers in the last 36 hours. All I can recommend is rebooting your computer, that will force Windows to start from scratch with the name servers and the rest of the software. Steve K4HG From cmowen at att.net Mon Jan 14 12:19:48 2008 From: cmowen at att.net (cmowen at att.net) Date: Mon, 14 Jan 2008 18:19:48 +0000 Subject: [wxqc] Some Stations down through APRS/Findu Message-ID: <011420081819.7218.478BA7C40008AE0A00001C3222230650029B0A02D29B9B0EBF020A9901030C@att.net> I am a remote to my computer and have done nothing to it, but it started working again at about noon: CW4870>APRS,TCPXX*,qAX,THIRD:@141705z Others, I saw at the same time, have come back except KG4GJX and one that looks intermittant anyway. I don't have an explanation. Charles -------------- Original message from Steve Dimse : -------------- > > On Jan 14, 2008, at 8:31 AM, cmowen at att.net wrote: > > > My station and others nearby have stopped getting to the findu site. > > But I see that several are still working so the whole system is not > > down. My Raw data suggests that I was sending to "Third" predomately > > right before the fail. Others that are down show alot of flip- > > flopping. Even the ones that are up seem to be going between "First" > > and "Fourth" alot. > > > > Any Ideas where the breakdown is? Data getting to APRS? MADIS? just > > not displaying? > > Ideally each stations should be flip-flopping. When your weather > software asks to connect to rotate.aprs.net, it is given a list of the > four collection hubs, in what is essentially a random order. Your > system is supposed to use the list in this order and not cache it. > Windows is notorious for not following the rules though, so there is > often unwanted caching of the data. > > I can't say specifically why you and your friends are having problems, > only that the system is up and functioning normally, with a normal > number of users sending data, there has been no drop in output numbers > in the last 36 hours. > > All I can recommend is rebooting your computer, that will force > Windows to start from scratch with the name servers and the rest of > the software. > > Steve K4HG > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. From gerry.creager at tamu.edu Mon Jan 14 12:54:31 2008 From: gerry.creager at tamu.edu (Gerry Creager) Date: Mon, 14 Jan 2008 12:54:31 -0600 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420081331.1922.478B6419000F0AE00000078222230704929B0A02D29B9B0EBF020A9901030C@att.net> References: <011420081331.1922.478B6419000F0AE00000078222230704929B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <478BAFE7.2080408@tamu.edu> There was some maintenance on the Core servers this weekend involving restarting various servers. first will do this sometime later today, too, I hope. As Steve Dimse posited, Windows doesn't work and play well with DNS refresh sometimes. That said, some of the Weather software developers have gone through the hoops to make their individual programs behave better (Thanks, guys!). Seeing the data bounce between First and Fourth, and/or Second and Third, isn't bad. We're all interconnected, and comms are passing amongst the Core systems nicely. So, consider this a heads-up that I'm replacing the Java code sometime today or tonight (or maybe tomorrow) and will reboot First at that time, causing a longer outage than if I just restart the Server. gerry cmowen at att.net wrote: > My station and others nearby have stopped getting to the findu site. But > I see that several are still working so the whole system is not down. My > Raw data suggests that I was sending to "Third" predomately right before > the fail. Others that are down show alot of flip-flopping. Even the ones > that are up seem to be going between "First" and "Fourth" alot. > > Any Ideas where the breakdown is? Data getting to APRS? MADIS? just not > displaying? > > Charles > CW4870 > > > ------------------------------------------------------------------------ > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. -- Gerry Creager -- gerry.creager at tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From cmowen at att.net Mon Jan 14 13:37:31 2008 From: cmowen at att.net (cmowen at att.net) Date: Mon, 14 Jan 2008 19:37:31 +0000 Subject: [wxqc] Some Stations down through APRS/Findu Message-ID: <011420081937.18890.478BB9FB000B6330000049CA22243322829B0A02D29B9B0EBF020A9901030C@att.net> Thanks. As of January 2008, I am running WeatherLink now and bounce between "First" and "Third" it seems. I previously ran WUHU which has "Rotate" plus three more slots to set up in case. I wish that I had noticed the Down at 5am to take a quick look at windows DNS info, but alas I looked only at what WL sends to my still "under construction" web-site: (http://owentech.com/wx). Hopefully, this will not happen again, but I will try to see if it is just "Win-Dumbs" casheing. Charles CW4870 www.owentech.com/wx -------------- Original message from Gerry Creager : -------------- > There was some maintenance on the Core servers this weekend involving > restarting various servers. first will do this sometime later today, > too, I hope. > > As Steve Dimse posited, Windows doesn't work and play well with DNS > refresh sometimes. That said, some of the Weather software developers > have gone through the hoops to make their individual programs behave > better (Thanks, guys!). > > Seeing the data bounce between First and Fourth, and/or Second and > Third, isn't bad. We're all interconnected, and comms are passing > amongst the Core systems nicely. > > So, consider this a heads-up that I'm replacing the Java code sometime > today or tonight (or maybe tomorrow) and will reboot First at that time, > causing a longer outage than if I just restart the Server. > > gerry > > cmowen at att.net wrote: > > My station and others nearby have stopped getting to the findu site. But > > I see that several are still working so the whole system is not down. My > > Raw data suggests that I was sending to "Third" predominately right before > > the fail. Others that are down show alot of flip-flopping. Even the ones > > that are up seem to be going between "First" and "Fourth" alot. > > > > Any Ideas where the breakdown is? Data getting to APRS? MADIS? just not > > displaying? > > > > Charles > > CW4870 > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > wxqc mailing list > > Post messages to wxqc at lists.gladstonefamily.net > > To unsubcribe or change delivery options, please go to: > > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > > > The contents of this message are the responsibility of the author. > > -- > Gerry Creager -- gerry.creager at tamu.edu > Texas Mesonet -- AATLT, Texas A&M University > Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983 > Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/549bfb05/attachment.html From gerry.creager at tamu.edu Mon Jan 14 13:46:06 2008 From: gerry.creager at tamu.edu (Gerry Creager) Date: Mon, 14 Jan 2008 13:46:06 -0600 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420081937.18890.478BB9FB000B6330000049CA22243322829B0A02D29B9B0EBF020A9901030C@att.net> References: <011420081937.18890.478BB9FB000B6330000049CA22243322829B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <478BBBFE.3060102@tamu.edu> See if you can tell it to use rotate instead of 'first' and 'third'. That's the preferred way to go. cmowen at att.net wrote: > Thanks. As of January 2008, I am running WeatherLink now and bounce > between "First" and "Third" it seems. I previously ran WUHU which has > "Rotate" plus three more slots to set up in case. I wish that I had > noticed the Down at 5am to take a quick look at windows DNS info, but > alas I looked only at what WL sends to my still "under construction" > web-site: (http://owentech.com/wx). > > > > Hopefully, this will not happen again, but I will try to see if it is > just "Win-Dumbs" casheing. > > > > > > Charles > > CW4870 > > www.owentech.com/wx > > > > > -------------- Original message from Gerry Creager > : -------------- > > > > There was some maintenance on the Core servers this weekend > involving > > restarting various servers. first will do this sometime later today, > > too, I hope. > > > > As Steve Dimse posited, Windows doesn't work and play well with DNS > > refresh sometimes. That said, some of the Weather software > developers > > have gone through the hoops to make their individual programs behave > > better (Thanks, guys!). > > > > Seeing the data bounce between First and Fourth, and/or Second and > > Third, isn't bad. We're all interconnected, and comms are passing > > amongst the Core systems nicely. > > > > So, consider this a heads-up that I'm replacing the Java code > sometime > > today or tonight (or maybe tomorrow) and will r eboot First at > that time, > > causing a longer outage than if I just restart the Server. > > > > gerry > > > > cmowen at att.net wrote: > > > My station and others nearby have stopped getting to the findu > site. But > > > I see that several are still working so the whole system is not > down. My > > > Raw data suggests that I was sending to "Third" predominately > right before > > > the fail. Others that are down show alot of flip-flopping. Even > the ones > > > that are up seem to be going between "First" and "Fourth" alot. > > > > > > Any Ideas where the breakdown is? Data getting to APRS? MADIS? > just not > > > displaying? > > > > > > Charles > > > CW4870 > > > > > > > > > > ------------------------------------------------------------------------ > > > > > > > _______________________________________________ > > > wxqc mailing lis t > > > Post messages to wxqc at lists.gladstonefamily.net > > > To unsubcribe or change delivery options, please go to: > > > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > > > > > The contents of this message are the responsibility of the author. > > > > -- > > Gerry Creager -- gerry.creager at tamu.edu > > Texas Mesonet -- AATLT, Texas A&M University > > Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983 > > Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX > 77843 > > > > _______________________________________________ > > wxqc mailing list > > Post messages to wxqc at lists.gladstonefamily.net > > To unsubcribe or change delivery options, please go to: > > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > > > The contents of this message are the responsibility of the author. > > > ------------------------------------------------------------------------ > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. -- Gerry Creager -- gerry.creager at tamu.edu Texas Mesonet -- AATLT, Texas A&M University Cell: 979.229.5301 Office: 979.862.3982 FAX: 979.862.3983 Office: 1700 Research Parkway Ste 160, TAMU, College Station, TX 77843 From dave at aprsfl.net Mon Jan 14 13:41:25 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Mon, 14 Jan 2008 14:41:25 -0500 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420081819.7218.478BA7C40008AE0A00001C3222230650029B0A02D29B9B0EBF020A9901030C@att.net> References: <011420081819.7218.478BA7C40008AE0A00001C3222230650029B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <038801c856e5$731a44a0$594ecde0$@net> Howdy.... Look at the attached URL: http://www.wxqa.com/checkservers.html You'll see if you watch this on a regular basis that traffic always bounces around between the servers a bit, but the total station count is always within 100 or so (not all stations update 24 hours a day, for example). 4400 is about where we've been all month, and is where we were during the time you had problems. It's a good possibility that the software you are using is -not- working with the rotate system correctly, and when there was net congestion to third that you never moved to the other servers. That being the case, make sure you have the latest of your software, we've gotten most of the developers (not all) to update their software to do this right. Alternatively, if you can list more than one server in your config, put first.aprs.net, second.aprs.net, third.aprs.net, and fourth.aprs.net (rotate is a "alias" for all of them, it's not actually a server) in your configuration -under- rotate.aprs.net just in case, that way if rotate totally fails on your PC, it'll manually go to the servers on its own. Also if you are running Windows XP, make sure you have Service Pack 2 installed or rotate will definitely have issues. You can read all about where the servers are, and look at their status pages when you think you have problems at http://core.aprs.net -- had you looked at third's status page at the time, you'd have gotten it as I did ever time right away showing over 200 amateur radio operators connected to it.... or it'd have taken 3-4 seconds for the status page to load (the net was congested for some users on the internet to that server for a few hours). The owners of the datacenter where third is said there was some very large fiber cut in the Houston area that caused part of the congestion, it wasn't "down", but running very slow until traffic was re-routed. Remember, when looking at the server status pages that amateur radio operators will stay connected to the server all the time, as we use APRS as a tactical real time network on the internet -and- radio across the country, where weather is only one of a slew of things we use aprs for (vehicle tracking with GPS, balloon tracking, tide level monitoring, two way messaging similar to instant messaging, etc). CWOP internet users only connect for a second to drop off your data and disconnect, so most of the time on the status page of a server --- even with 4000+ CWOP users worldwide --- you'll see about 20-50 CW calls at any given time on any one server... More on the "5's" of the clock (I wish developers didn't use actual time for the submissions). There may be another problem we'll have to approach with developers, if they are only waiting a second once they connect to a server to submit the data before "giving up", that's not long enough to account for problems that can arise with fiber cable cuts, high net traffic, etc.... If this was the case vs the rotate problem, you would not have noticed this if the software waited a a bit longer, as most amateur radio operators didn't as our software is designed for long waits on radio too, so we just experienced a 2-3 second lag with software like UI-View. At any rate, the fiber cut has been band-aided and if you couldn't get to third, you should be able to now... But be aware, we run more than one server for a reason, there are times we have to take them down for maintenance, and if you use rotate correctly or move around the servers as you should, you'll never see any downtime. Essentially what it boils down to... if you had problems today, you need to address them with your software before the next problem or maintenance window occurs, that way you'll not experience downtime again. Hope this helps with the understanding of the system. Seeya, Dave KG4YZY http://www.aprsfl.net Sysop of APRSFL, THIRD, and FOURTH -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of cmowen at att.net Sent: Monday, January 14, 2008 1:20 PM To: Discussion of weather data quality issues Subject: Re: [wxqc] Some Stations down through APRS/Findu I am a remote to my computer and have done nothing to it, but it started working again at about noon: CW4870>APRS,TCPXX*,qAX,THIRD:@141705z Others, I saw at the same time, have come back except KG4GJX and one that looks intermittant anyway. I don't have an explanation. Charles -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From dave at aprsfl.net Mon Jan 14 12:59:43 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Mon, 14 Jan 2008 13:59:43 -0500 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420081819.7218.478BA7C40008AE0A00001C3222230650029B0A02D29B9B0EBF020A9901030C@att.net> References: <011420081819.7218.478BA7C40008AE0A00001C3222230650029B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <036a01c856df$9fcbf9e0$df63eda0$@net> Howdy.... Look at the attached URL: http://www.wxqa.com/checkservers.html You'll see if you watch this on a regular basis that traffic always bounces around between the servers a bit, but the total station count is always within 100 or so (not all stations update 24 hours a day, for example). 4400 is about where we've been all month, and is where we were during the time you had problems. It's a good possibility that the software you are using is -not- working with the rotate system correctly, and when there was net congestion to third that you never moved to the other servers. That being the case, make sure you have the latest of your software, we've gotten most of the developers (not all) to update their software to do this right. Alternatively, if you can list more than one server in your config, put first.aprs.net, second.aprs.net, third.aprs.net, and fourth.aprs.net (rotate is a "alias" for all of them, it's not actually a server) in your configuration -under- rotate.aprs.net just in case, that way if rotate totally fails on your PC, it'll manually go to the servers on its own. Also if you are running Windows XP, make sure you have Service Pack 2 installed or rotate will definitely have issues. You can read all about where the servers are, and look at their status pages when you think you have problems at http://core.aprs.net -- had you looked at third's status page at the time, you'd have gotten it as I did ever time right away showing over 200 amateur radio operators connected to it.... or it'd have taken 3-4 seconds for the status page to load (the net was congested for some users on the internet to that server for a few hours). The owners of the datacenter where third is said there was some very large fiber cut in the Houston area that caused part of the congestion, it wasn't "down", but running very slow until traffic was re-routed. Remember, when looking at the server status pages that amateur radio operators will stay connected to the server all the time, as we use APRS as a tactical real time network on the internet -and- radio across the country, where weather is only one of a slew of things we use aprs for (vehicle tracking with GPS, balloon tracking, tide level monitoring, two way messaging similar to instant messaging, etc). CWOP internet users only connect for a second to drop off your data and disconnect, so most of the time on the status page of a server --- even with 4000+ CWOP users worldwide --- you'll see about 20-50 CW calls at any given time on any one server... More on the "5's" of the clock (I wish developers didn't use actual time for the submissions). There may be another problem we'll have to approach with developers, if they are only waiting a second once they connect to a server to submit the data before "giving up", that's not long enough to account for problems that can arise with fiber cable cuts, high net traffic, etc.... If this was the case vs the rotate problem, you would not have noticed this if the software waited a a bit longer, as most amateur radio operators didn't as our software is designed for long waits on radio too, so we just experienced a 2-3 second lag with software like UI-View. At any rate, the fiber cut has been band-aided and if you couldn't get to third, you should be able to now... But be aware, we run more than one server for a reason, there are times we have to take them down for maintenance, and if you use rotate correctly or move around the servers as you should, you'll never see any downtime. Essentially what it boils down to... if you had problems today, you need to address them with your software before the next problem or maintenance window occurs, that way you'll not experience downtime again. Hope this helps with the understanding of the system. Seeya, Dave KG4YZY http://www.aprsfl.net Sysop of APRSFL, THIRD, and FOURTH -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of cmowen at att.net Sent: Monday, January 14, 2008 1:20 PM To: Discussion of weather data quality issues Subject: Re: [wxqc] Some Stations down through APRS/Findu I am a remote to my computer and have done nothing to it, but it started working again at about noon: CW4870>APRS,TCPXX*,qAX,THIRD:@141705z Others, I saw at the same time, have come back except KG4GJX and one that looks intermittant anyway. I don't have an explanation. Charles -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. From cmowen at att.net Mon Jan 14 14:46:03 2008 From: cmowen at att.net (cmowen at att.net) Date: Mon, 14 Jan 2008 20:46:03 +0000 Subject: [wxqc] Some Stations down through APRS/Findu Message-ID: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net> Dave (and et al), First of all, I did not mean to sound like I was attacking "Third" in any way. I work in IT and answer the phone when potentially 8000 users can't do their work. I understand when "networking" tells me it is not us that is the problem. I also don't want to have a "gripe-in" like I saw in the lists two years ago when I joined. You guys do a monumental job. Anyway, I have just noticed the I float between "First" and "Third" on visual inspection of what "Rotate" does for me. I just started using the Davis Weatherlink software and I think that version 5.71 is the latest. It is using the "Rotate" scheme. This is the first time I have seen a problem seemed "not at my end". When WUHU failed to send, it was either my old station or computer acting up and nobody got data. (They allow explicitly 4 servers to be listed to try). I was still sending through WeatherLink today to Weather Underground and my ftp site. Therefore, if WL571 is using "Rotate", I have to look at Windows2003 (fully updated), DNS issues from Charter, WAN router, or space aliens to be determined. It is interesting to note from your weblink, that the incident started about 0800UTC today and that "Third"'s load seeming all shifted to "Fourth". "Fisrt" picked up some re-distributed load, but it seemed to be more of a sequential shift to "Fourth" than a load balance. Hour(UTC)FIRSTSECONDTHIRDFOURTHOtherTotal 1927017221878629644511 1826217422318369604519 1730417021408729634506 1645320041723629704460 1543520042523559674438 1444319243323299564411 1343817841723549504394 1244019541423149444364 1141220745122839364346 1030918219399579104350 09322173173710768594222 0830816319559368974310 0724216421708179184362 0625216221338559154368 0525116221468389104361 0426017021518359204388 0327317521388409294408 0225617221748489384438 0126117122008209474451 Charles CW4870 http://www.owentech.com/wx -------------- Original message from "Dave Anderson KG4YZY" : -------------- > Howdy.... > > Look at the attached URL: > > http://www.wxqa.com/checkservers.html > > You'll see if you watch this on a regular basis that traffic always bounces > around between the servers a bit, but the total station count is always > within 100 or so (not all stations update 24 hours a day, for example). > 4400 is about where we've been all month, and is where we were during the > time you had problems. > > It's a good possibility that the software you are using is -not- working > with the rotate system correctly, and when there was net congestion to third > that you never moved to the other servers. That being the case, make sure > you have the latest of your software, we've gotten most of the developers > (not all) to update their software to do this right. > > Alternatively, if you can list more than one server in your config, put > first.aprs.net, second.aprs.net, third.aprs.net, and fourth.aprs.net > (rotate is a "alias" for all of them, it's not actually a server) in your > configuration -under- rotate.aprs.net just in case, that way if rotate > totally fails on your PC, it'll manually go to the servers on its own. > Also if you are running Windows XP, make sure you have Service Pack 2 > installed or rotate will definitely have issues. > > You can read all about where the servers are, and look at their status pages > when you think you have problems at http://core.aprs.net -- had you looked > at third's status page at the time, you'd have gotten it as I did ever time > right away showing over 200 amateur radio operators connected to it.... or > it'd have taken 3-4 seconds for the status page to load (the net was > congested for some users on the internet to that server for a few hours). > The owners of the datacenter where third is said there was some very large > fiber cut in the Houston area that caused part of the congestion, it wasn't > "down", but running very slow until traffic was re-routed. > > Remember, when looking at the server status pages that amateur radio > operators will stay connected to the server all the time, as we use APRS as > a tactical real time network on the internet -and- radio across the country, > where weather is only one of a slew of things we use aprs for (vehicle > tracking with GPS, balloon tracking, tide level monitoring, two way > messaging similar to instant messaging, etc). > > CWOP internet users only connect for a second to drop off your data and > disconnect, so most of the time on the status page of a server --- even with > 4000+ CWOP users worldwide --- you'll see about 20-50 CW calls at any given > time on any one server... More on the "5's" of the clock (I wish developers > didn't use actual time for the submissions). > > There may be another problem we'll have to approach with developers, if they > are only waiting a second once they connect to a server to submit the data > before "giving up", that's not long enough to account for problems that can > arise with fiber cable cuts, high net traffic, etc.... > > If this was the case vs the rotate problem, you would not have noticed this > if the software waited a a bit longer, as most amateur radio operators > didn't as our software is designed for long waits on radio too, so we just > experienced a 2-3 second lag with software like UI-View. > > At any rate, the fiber cut has been band-aided and if you couldn't get to > third, you should be able to now... But be aware, we run more than one > server for a reason, there are times we have to take them down for > maintenance, and if you use rotate correctly or move around the servers as > you should, you'll never see any downtime. > > Essentially what it boils down to... if you had problems today, you need to > address them with your software before the next problem or maintenance > window occurs, that way you'll not experience downtime again. > > Hope this helps with the understanding of the system. > > Seeya, > Dave > KG4YZY > http://www.aprsfl.net > Sysop of APRSFL, THIRD, and FOURTH > > > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of cmowen at att.net > Sent: Monday, January 14, 2008 1:20 PM > To: Discussion of weather data quality issues > Subject: Re: [wxqc] Some Stations down through APRS/Findu > > I am a remote to my computer and have done nothing to it, but it started > working again at about noon: > > CW4870>APRS,TCPXX*,qAX,THIRD:@141705z > > Others, I saw at the same time, have come back except KG4GJX and one that > looks intermittant anyway. > > I don't have an explanation. > > > Charles > > > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/ab695669/attachment.html From dave at fab-corp.com Mon Jan 14 18:13:11 2008 From: dave at fab-corp.com (Dave Anderson) Date: Mon, 14 Jan 2008 19:13:11 -0500 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net> References: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <044501c8570b$6a342c40$3e9c84c0$@com> First of all, I did not mean to sound like I was attacking "Third" in any way. I work in IT and answer the phone when potentially 8000 users can't do their work. I understand when "networking" tells me it is not us that is the problem. I also don't want to have a "gripe-in" like I saw in the lists two years ago when I joined. You guys do a monumental job. Wasn't taken as an attack, it'd been a while since I posted my soapbox how the servers run message for new folks who've recently joined, so it worked out ok. A lot of CWOP users don't realize that APRS is actually an amateur radio network J. I as well as you never wanted to see what happened 2 years ago. Unfortunately defending ourselves proved to give us a black eye. The three of us do nothing but work on internet backbone issues and massive server farms all day, we knew what our networks could do, and when told otherwise by outsiders it was a very personal pride issue for myself, to be specific. Ultimately all the "doom and gloom" was proven wrong, as we knew it would be, and things have been working great since. I appreciate your comments.... The three of us do this only for the love of the hobby, nothing more. And with quad CPU xeon boxes in my case and 10+MB/s of internet that the APRS-IS at the core represents between the 4 servers, it's a rather costly hobby for us J Anyway, I have just noticed the I float between "First" and "Third" on visual inspection of what "Rotate" does for me. I just started using the Davis Weatherlink software and I think that version 5.71 is the latest. It is using the "Rotate" scheme. This is the first time I have seen a problem seemed "not at my end". When WUHU failed to send, it was either my old station or computer acting up and nobody got data. (They allow explicitly 4 servers to be listed to try). I was still sending through WeatherLink today to Weather Underground and my ftp site. That's odd. I wonder why you never go to second and fourth. If I sit here and do a DNS resolve on rotate, I get all four of them, and the TTL's are all VERY low and match, so there's no reason from a DNS perspective that "one" wins more than the other. I have a copy of the WL that I got with my VP2 here at KG4YZY (I use a ham APRS program for submitting data, as we also transmit our weather on air so others can see it, during disasters with weather, the net is down, but 2 meter ham radio still works). I'll have to install it on my laptop and see if I can figure out what it's up to. Therefore, if WL571 is using "Rotate", I have to look at Windows2003 (fully updated), DNS issues from Charter, WAN router, or space aliens to be determined. Rotate is nothing more than 4 A records on the same name with matching TTL's. Pseudo round robin lookups -should- work. Some caching DNS servers don't' follow the RFC guidelines for TTL though. We run TTL's VERY low so if a box goes down or we have to do maintenance, we simply take it out of the DNS for the time we work on it, so folks go elsewhere. I'll post another message here in a bit with some tests I'll perform here showing that rotate does indeed work if the software does a fresh gethostname lookup with a negative local cache of it. It is interesting to note from your weblink, that the incident started about 0800UTC today and that "Third"'s load seeming all shifted to "Fourth". "Fisrt" picked up some re-distributed load, but it seemed to be more of a sequential shift to "Fourth" than a load balance. The fact that the load is as skewed as bad as it is always has made me wonder what's going on. I run other internet services with DNS systems like this, and if you look at logs, it'll balance out rather nicely. Almost makes me wonder if some off the software "prefers" some servers (not truly performing a fresh DNS lookup every connect)... Seeya, Dave KG4YZY www.aprsfl.net -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/21579a59/attachment-0001.html From dave at aprsfl.net Mon Jan 14 18:13:29 2008 From: dave at aprsfl.net (Dave Anderson KG4YZY) Date: Mon, 14 Jan 2008 19:13:29 -0500 Subject: [wxqc] How rotate.aprs.net is supposed to work.. In-Reply-To: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net> References: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net> Message-ID: <044a01c8570b$7486d7b0$5d948710$@net> If you don't care about networking.. or now load balancing the network for CWOP works, then this is your warning that this will be technical in nature, and not weather related, but it does affect the program, so I'm posting it. J If you don't care about this, then just delete this message.. If however you are curious, or work in the IT or ISP industry, here's my thoughts and observations on rotate.aprs.net's functionality. Rotate.aprs.net is nothing more than a DNS record with 4 A records on it for those of you who have experience with DNS. It has a VERY low TTL as well. The primary DNS server that runs APRS is on Findu, so I'm not doing these tests on the same network or server that Findu is on. My local DNS server is the standard run of the mill Bind installation on a SunOS box, no different that the lion share of ISP's run for customers to use. Now, on a different machine on my network, a CentOS5 box, here is what happens if I ping rotate.aprs.net over and over: ldm:~# ping rotate.aprs.net PING rotate.aprs.net (134.173.254.38) 56(84) bytes of data. 64 bytes from SoCal.APRS.Net (134.173.254.38): icmp_seq=0 ttl=54 time=75.5 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 75.527/75.527/75.527/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (67.18.222.130) 56(84) bytes of data. 64 bytes from 82.de.1243.static.theplanet.com (67.18.222.130): icmp_seq=0 ttl=52 time=37.9 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 37.936/37.936/37.936/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (64.58.200.120) 56(84) bytes of data. 64 bytes from fourth.aprsfl.net (64.58.200.120): icmp_seq=0 ttl=128 time=2.19 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 2.196/2.196/2.196/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (165.91.140.28) 56(84) bytes of data. 64 bytes from aprs.tamu.edu (165.91.140.28): icmp_seq=0 ttl=47 time=51.9 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 51.923/51.923/51.923/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (64.58.200.120) 56(84) bytes of data. 64 bytes from fourth.aprsfl.net (64.58.200.120): icmp_seq=0 ttl=128 time=0.224 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.224/0.224/0.224/0.000 ms, pipe 2 If you notice every time I did the ping, it did nothing more than a simple gethostname lookup and pinged the IP it resolved. Note it's not in order, it's random, but no matter, every server in 5 pings was hit at least once. No matter if I run this ping on a Linux box or a Windows XP or Vista box, I get the same results every time. What does this mean? Well, many things can mess up DNS caching. First and foremost is a ISP that is running a server overriding the TTL rules we put in place to "save bandwidth" on their network. Not many do this, but we have found a few. Usually users of those ISP's have problems with more than just CWOP though, they'll find times where they can't get to sites say such as CNN.com, because, as you probably guessed it, they also use multiple A records: ldm:~# nslookup www.cnn.com Server: 64.58.200.1 Address: 64.58.200.1#53 Non-authoritative answer: Name: www.cnn.com Address: 64.236.91.21 Name: www.cnn.com Address: 64.236.91.22 Name: www.cnn.com Address: 64.236.91.23 Name: www.cnn.com Address: 64.236.91.24 Name: www.cnn.com Address: 64.236.16.20 Name: www.cnn.com Address: 64.236.16.52 Name: www.cnn.com Address: 64.236.24.12 Name: www.cnn.com Address: 64.236.29.120 Now just as with rotate.aprs.net, every time you go to ping it, you hit a different server. Other issues we've found that can mess with DNS caching are broadband routers with "little built in" DNS servers. My advice is not to use them. Use your ISP's DNS server, unless you run a local copy of bind on Linux or DNS on Windows and talk to the root servers yourself. You'll be much happier with the results. The biggest problem we see to date though is folks who have not updated their CWOP software to the latest version that may correct this. Many of the developers worked overtime to get versions out that do work correctly with round robin DNS, and we thank them for that. There are a few apps that still do not do a fresh lookup every time you call rotate, and those programs will truly get hung on one server till you reload - or in some cases reboot the PC, hopefully those developers of software that does not support this system correctly reading this will realize the severity of this problem and work with us to fix it. There -are- a few who have reported that Windows itself causes this problem, but I have yet to replicate the problem on a service pack 2 XP machine or Vista. Some pre-service pack 2 installs of XP do get "dumb" on the cache, but still only hold the record for an hour if memory serves. At any rate, some folks make DNS sound more complicated than it really is. I've owned and operated a ISP since 1994. Yes, look at your calendar, you'll find that CERN hadn't even released a non beta of the HTTP protocol at that time, it was still gopher and archive for doing lookups online. It isn't for everyone to understand, but overall, it's not that incredibly complicated. Is round robin DNS the ideal and perfect solution? No. I run load balancing boxes using several technologies that work ten times better, but non will work with the specs of APRS. Trust me, we've looked into it J. If APRS-IS was UDP, then ANYCAST with BGP (how DNS servers load balance themselves) would be the way to go, but again, it's not a solution that will work for us. I wish I could provide more "insight" into this, but I've already overstayed my welcome for a tech topic such as this in the QC forum. If you have further questions about the APRS-IS network, go to http://core.aprs.net - you can e-mail any of the three of us that run the 4 servers, we'd be happy to answer any questions you may have. Developers with comments or questions please contact us! J Seeya, Dave KG4YZY www.aprsfl.net -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/48726b12/attachment-0001.html From k8cyj at arrl.net Mon Jan 14 20:53:11 2008 From: k8cyj at arrl.net (Rex Pirkle) Date: Mon, 14 Jan 2008 20:53:11 -0600 Subject: [wxqc] Some Stations down through APRS/Findu In-Reply-To: Message-ID: <000c01c85721$c80d47f0$6401a8c0@DJSSJ661> Oh good, an opportunity to engage in one of my favorite pastimes: Bashing Davis WeatherLink software. Even though the latest Davis software, "WeatherLink for Windows v 5.7.1" sets the default server to "rotate.aprs.net" port 14580, it does not appear to rotate constantly through the various servers. In my experience, it establishes a connection and re-uses that same server until encountering an upload error, or until a program restart occurs. In WeatherLink, the suggestion of adding multiple servers to the config, such as "first.aprs.net", "second.aprs.net", etc. will not work. Weatherlink will accept only *ONE* APRS Server within the entire Weatherlink program -- changing an APRS Telenet Server in one Profile causes the last entry to appear in *ALL* the Profiles. Restarting WL periodically to force a rotate isn't a good option for two reasons: (1) The program require a manual intervention to 'Start the weather bulletin' and (2) Restarting the program causes Pxxx 'Rain since Midnight' and pxxx 'Daily Rain' reports to disappear. The program monitors and sums those values in real-time, so it takes a full day to rebuild the values. Until the values are resolved, it reports 'p...' and 'P...'. I've tried various workarounds to circumvent these problems -- none of them successful. Reports of these issues have been made to Davis to no avail. It seems the only solution is to run second source software or put up with the missing data posts. Maybe if enough of us complain, they will get rid of that rehashed Windows 3.1 GUI and fix some of their problems. 73 Rex Pirkle - K8CYJ-5/AR863 http://weather.gladstonefamily.net/site/AR863 -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of wxqc-request at lists.gladstonefamily.net Sent: Monday, January 14, 2008 6:13 PM To: wxqc at lists.gladstonefamily.net Subject: wxqc Digest, Vol 39, Issue 6 Send wxqc mailing list submissions to wxqc at lists.gladstonefamily.net To subscribe or unsubscribe via the World Wide Web, visit http://server.gladstonefamily.net/mailman/listinfo/wxqc or, via email, send a message with subject or body 'help' to wxqc-request at lists.gladstonefamily.net You can reach the person managing the list at wxqc-owner at lists.gladstonefamily.net When replying, please edit your Subject line so it is more specific than "Re: Contents of wxqc digest..." Today's Topics: 1. Re: Some Stations down through APRS/Findu (Dave Anderson KG4YZY) 2. Re: Some Stations down through APRS/Findu (cmowen at att.net) 3. Re: Some Stations down through APRS/Findu (Dave Anderson) ---------------------------------------------------------------------- Message: 1 Date: Mon, 14 Jan 2008 13:59:43 -0500 From: "Dave Anderson KG4YZY" Subject: Re: [wxqc] Some Stations down through APRS/Findu To: "'Discussion of weather data quality issues'" Message-ID: <036a01c856df$9fcbf9e0$df63eda0$@net> Content-Type: text/plain; charset="us-ascii" Howdy.... Look at the attached URL: http://www.wxqa.com/checkservers.html You'll see if you watch this on a regular basis that traffic always bounces around between the servers a bit, but the total station count is always within 100 or so (not all stations update 24 hours a day, for example). 4400 is about where we've been all month, and is where we were during the time you had problems. It's a good possibility that the software you are using is -not- working with the rotate system correctly, and when there was net congestion to third that you never moved to the other servers. That being the case, make sure you have the latest of your software, we've gotten most of the developers (not all) to update their software to do this right. Alternatively, if you can list more than one server in your config, put first.aprs.net, second.aprs.net, third.aprs.net, and fourth.aprs.net (rotate is a "alias" for all of them, it's not actually a server) in your configuration -under- rotate.aprs.net just in case, that way if rotate totally fails on your PC, it'll manually go to the servers on its own. Also if you are running Windows XP, make sure you have Service Pack 2 installed or rotate will definitely have issues. You can read all about where the servers are, and look at their status pages when you think you have problems at http://core.aprs.net -- had you looked at third's status page at the time, you'd have gotten it as I did ever time right away showing over 200 amateur radio operators connected to it.... or it'd have taken 3-4 seconds for the status page to load (the net was congested for some users on the internet to that server for a few hours). The owners of the datacenter where third is said there was some very large fiber cut in the Houston area that caused part of the congestion, it wasn't "down", but running very slow until traffic was re-routed. Remember, when looking at the server status pages that amateur radio operators will stay connected to the server all the time, as we use APRS as a tactical real time network on the internet -and- radio across the country, where weather is only one of a slew of things we use aprs for (vehicle tracking with GPS, balloon tracking, tide level monitoring, two way messaging similar to instant messaging, etc). CWOP internet users only connect for a second to drop off your data and disconnect, so most of the time on the status page of a server --- even with 4000+ CWOP users worldwide --- you'll see about 20-50 CW calls at any given time on any one server... More on the "5's" of the clock (I wish developers didn't use actual time for the submissions). There may be another problem we'll have to approach with developers, if they are only waiting a second once they connect to a server to submit the data before "giving up", that's not long enough to account for problems that can arise with fiber cable cuts, high net traffic, etc.... If this was the case vs the rotate problem, you would not have noticed this if the software waited a a bit longer, as most amateur radio operators didn't as our software is designed for long waits on radio too, so we just experienced a 2-3 second lag with software like UI-View. At any rate, the fiber cut has been band-aided and if you couldn't get to third, you should be able to now... But be aware, we run more than one server for a reason, there are times we have to take them down for maintenance, and if you use rotate correctly or move around the servers as you should, you'll never see any downtime. Essentially what it boils down to... if you had problems today, you need to address them with your software before the next problem or maintenance window occurs, that way you'll not experience downtime again. Hope this helps with the understanding of the system. Seeya, Dave KG4YZY http://www.aprsfl.net Sysop of APRSFL, THIRD, and FOURTH -----Original Message----- From: wxqc-bounces at lists.gladstonefamily.net [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of cmowen at att.net Sent: Monday, January 14, 2008 1:20 PM To: Discussion of weather data quality issues Subject: Re: [wxqc] Some Stations down through APRS/Findu I am a remote to my computer and have done nothing to it, but it started working again at about noon: CW4870>APRS,TCPXX*,qAX,THIRD:@141705z Others, I saw at the same time, have come back except KG4GJX and one that looks intermittant anyway. I don't have an explanation. Charles -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. ------------------------------ Message: 2 Date: Mon, 14 Jan 2008 20:46:03 +0000 From: cmowen at att.net Subject: Re: [wxqc] Some Stations down through APRS/Findu To: Discussion of weather data quality issues Message-ID: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020 A9901030C at att.net> Content-Type: text/plain; charset="us-ascii" Dave (and et al), First of all, I did not mean to sound like I was attacking "Third" in any way. I work in IT and answer the phone when potentially 8000 users can't do their work. I understand when "networking" tells me it is not us that is the problem. I also don't want to have a "gripe-in" like I saw in the lists two years ago when I joined. You guys do a monumental job. Anyway, I have just noticed the I float between "First" and "Third" on visual inspection of what "Rotate" does for me. I just started using the Davis Weatherlink software and I think that version 5.71 is the latest. It is using the "Rotate" scheme. This is the first time I have seen a problem seemed "not at my end". When WUHU failed to send, it was either my old station or computer acting up and nobody got data. (They allow explicitly 4 servers to be listed to try). I was still sending through WeatherLink today to Weather Underground and my ftp site. Therefore, if WL571 is using "Rotate", I have to look at Windows2003 (fully updated), DNS issues from Charter, WAN router, or space aliens to be determined. It is interesting to note from your weblink, that the incident started about 0800UTC today and that "Third"'s load seeming all shifted to "Fourth". "Fisrt" picked up some re-distributed load, but it seemed to be more of a sequential shift to "Fourth" than a load balance. Hour(UTC)FIRSTSECONDTHIRDFOURTHOtherTotal 1927017221878629644511 1826217422318369604519 1730417021408729634506 1645320041723629704460 1543520042523559674438 1444319243323299564411 1343817841723549504394 1244019541423149444364 1141220745122839364346 1030918219399579104350 09322173173710768594222 0830816319559368974310 0724216421708179184362 0625216221338559154368 0525116221468389104361 0426017021518359204388 0327317521388409294408 0225617221748489384438 0126117122008209474451 Charles CW4870 http://www.owentech.com/wx -------------- Original message from "Dave Anderson KG4YZY" : -------------- > Howdy.... > > Look at the attached URL: > > http://www.wxqa.com/checkservers.html > > You'll see if you watch this on a regular basis that traffic always bounces > around between the servers a bit, but the total station count is always > within 100 or so (not all stations update 24 hours a day, for example). > 4400 is about where we've been all month, and is where we were during the > time you had problems. > > It's a good possibility that the software you are using is -not- working > with the rotate system correctly, and when there was net congestion to third > that you never moved to the other servers. That being the case, make sure > you have the latest of your software, we've gotten most of the developers > (not all) to update their software to do this right. > > Alternatively, if you can list more than one server in your config, put > first.aprs.net, second.aprs.net, third.aprs.net, and fourth.aprs.net > (rotate is a "alias" for all of them, it's not actually a server) in your > configuration -under- rotate.aprs.net just in case, that way if rotate > totally fails on your PC, it'll manually go to the servers on its own. > Also if you are running Windows XP, make sure you have Service Pack 2 > installed or rotate will definitely have issues. > > You can read all about where the servers are, and look at their status pages > when you think you have problems at http://core.aprs.net -- had you looked > at third's status page at the time, you'd have gotten it as I did ever time > right away showing over 200 amateur radio operators connected to it.... or > it'd have taken 3-4 seconds for the status page to load (the net was > congested for some users on the internet to that server for a few hours). > The owners of the datacenter where third is said there was some very large > fiber cut in the Houston area that caused part of the congestion, it wasn't > "down", but running very slow until traffic was re-routed. > > Remember, when looking at the server status pages that amateur radio > operators will stay connected to the server all the time, as we use APRS as > a tactical real time network on the internet -and- radio across the country, > where weather is only one of a slew of things we use aprs for (vehicle > tracking with GPS, balloon tracking, tide level monitoring, two way > messaging similar to instant messaging, etc). > > CWOP internet users only connect for a second to drop off your data and > disconnect, so most of the time on the status page of a server --- even with > 4000+ CWOP users worldwide --- you'll see about 20-50 CW calls at any given > time on any one server... More on the "5's" of the clock (I wish developers > didn't use actual time for the submissions). > > There may be another problem we'll have to approach with developers, if they > are only waiting a second once they connect to a server to submit the data > before "giving up", that's not long enough to account for problems that can > arise with fiber cable cuts, high net traffic, etc.... > > If this was the case vs the rotate problem, you would not have noticed this > if the software waited a a bit longer, as most amateur radio operators > didn't as our software is designed for long waits on radio too, so we just > experienced a 2-3 second lag with software like UI-View. > > At any rate, the fiber cut has been band-aided and if you couldn't get to > third, you should be able to now... But be aware, we run more than one > server for a reason, there are times we have to take them down for > maintenance, and if you use rotate correctly or move around the servers as > you should, you'll never see any downtime. > > Essentially what it boils down to... if you had problems today, you need to > address them with your software before the next problem or maintenance > window occurs, that way you'll not experience downtime again. > > Hope this helps with the understanding of the system. > > Seeya, > Dave > KG4YZY > http://www.aprsfl.net > Sysop of APRSFL, THIRD, and FOURTH > > > -----Original Message----- > From: wxqc-bounces at lists.gladstonefamily.net > [mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of cmowen at att.net > Sent: Monday, January 14, 2008 1:20 PM > To: Discussion of weather data quality issues > Subject: Re: [wxqc] Some Stations down through APRS/Findu > > I am a remote to my computer and have done nothing to it, but it started > working again at about noon: > > CW4870>APRS,TCPXX*,qAX,THIRD:@141705z > > Others, I saw at the same time, have come back except KG4GJX and one that > looks intermittant anyway. > > I don't have an explanation. > > > Charles > > > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > _______________________________________________ > wxqc mailing list > Post messages to wxqc at lists.gladstonefamily.net > To unsubcribe or change delivery options, please go to: > http://server.gladstonefamily.net/mailman/listinfo/wxqc > > The contents of this message are the responsibility of the author. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/ab 695669/attachment-0001.html ------------------------------ Message: 3 Date: Mon, 14 Jan 2008 19:13:11 -0500 From: "Dave Anderson" Subject: Re: [wxqc] Some Stations down through APRS/Findu To: "'Discussion of weather data quality issues'" Message-ID: <044501c8570b$6a342c40$3e9c84c0$@com> Content-Type: text/plain; charset="us-ascii" First of all, I did not mean to sound like I was attacking "Third" in any way. I work in IT and answer the phone when potentially 8000 users can't do their work. I understand when "networking" tells me it is not us that is the problem. I also don't want to have a "gripe-in" like I saw in the lists two years ago when I joined. You guys do a monumental job. Wasn't taken as an attack, it'd been a while since I posted my soapbox how the servers run message for new folks who've recently joined, so it worked out ok. A lot of CWOP users don't realize that APRS is actually an amateur radio network J. I as well as you never wanted to see what happened 2 years ago. Unfortunately defending ourselves proved to give us a black eye. The three of us do nothing but work on internet backbone issues and massive server farms all day, we knew what our networks could do, and when told otherwise by outsiders it was a very personal pride issue for myself, to be specific. Ultimately all the "doom and gloom" was proven wrong, as we knew it would be, and things have been working great since. I appreciate your comments.... The three of us do this only for the love of the hobby, nothing more. And with quad CPU xeon boxes in my case and 10+MB/s of internet that the APRS-IS at the core represents between the 4 servers, it's a rather costly hobby for us J Anyway, I have just noticed the I float between "First" and "Third" on visual inspection of what "Rotate" does for me. I just started using the Davis Weatherlink software and I think that version 5.71 is the latest. It is using the "Rotate" scheme. This is the first time I have seen a problem seemed "not at my end". When WUHU failed to send, it was either my old station or computer acting up and nobody got data. (They allow explicitly 4 servers to be listed to try). I was still sending through WeatherLink today to Weather Underground and my ftp site. That's odd. I wonder why you never go to second and fourth. If I sit here and do a DNS resolve on rotate, I get all four of them, and the TTL's are all VERY low and match, so there's no reason from a DNS perspective that "one" wins more than the other. I have a copy of the WL that I got with my VP2 here at KG4YZY (I use a ham APRS program for submitting data, as we also transmit our weather on air so others can see it, during disasters with weather, the net is down, but 2 meter ham radio still works). I'll have to install it on my laptop and see if I can figure out what it's up to. Therefore, if WL571 is using "Rotate", I have to look at Windows2003 (fully updated), DNS issues from Charter, WAN router, or space aliens to be determined. Rotate is nothing more than 4 A records on the same name with matching TTL's. Pseudo round robin lookups -should- work. Some caching DNS servers don't' follow the RFC guidelines for TTL though. We run TTL's VERY low so if a box goes down or we have to do maintenance, we simply take it out of the DNS for the time we work on it, so folks go elsewhere. I'll post another message here in a bit with some tests I'll perform here showing that rotate does indeed work if the software does a fresh gethostname lookup with a negative local cache of it. It is interesting to note from your weblink, that the incident started about 0800UTC today and that "Third"'s load seeming all shifted to "Fourth". "Fisrt" picked up some re-distributed load, but it seemed to be more of a sequential shift to "Fourth" than a load balance. The fact that the load is as skewed as bad as it is always has made me wonder what's going on. I run other internet services with DNS systems like this, and if you look at logs, it'll balance out rather nicely. Almost makes me wonder if some off the software "prefers" some servers (not truly performing a fresh DNS lookup every connect)... Seeya, Dave KG4YZY www.aprsfl.net -- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean. -------------- next part -------------- An HTML attachment was scrubbed... URL: http://server.gladstonefamily.net/pipermail/wxqc/attachments/20080114/21 579a59/attachment.html ------------------------------ _______________________________________________ wxqc mailing list Send messages to wxqc at lists.gladstonefamily.net To unsubscribe or change options, please go to: http://server.gladstonefamily.net/mailman/listinfo/wxqc The contents of these messages are the responsibility of the author(s). End of wxqc Digest, Vol 39, Issue 6 *********************************** From shell at shellware.com Mon Jan 14 22:03:17 2008 From: shell at shellware.com (Shell Shrader) Date: Mon, 14 Jan 2008 23:03:17 -0500 Subject: [wxqc] How rotate.aprs.net is supposed to work.. In-Reply-To: <044a01c8570b$7486d7b0$5d948710$@net> References: <011420082046.7185.478BCA0B0007BE2100001C1122243651029B0A02D29B9B0EBF020A9901030C@att.net>, <044a01c8570b$7486d7b0$5d948710$@net> Message-ID: Hi Dave, I always enjoy technical discussions with ya'll. In regards to: There -are- a few who have reported that Windows itself causes this problem, but I have yet to replicate the problem on a service pack 2 XP machine or Vista. Some pre-service pack 2 installs of XP do get "dumb" on the cache, but still only hold the record for an hour if memory serves. HI believe m$ fixed this prior to XP SP2. You're going to have to go back a bit farther to replicate the problem. It's been a few years but if memory serves correctly (I'm severely over-simplifying this) Windows used to arbitrarily cache DNS... sometimes the only way to flush the cache was with an explicit ipconfig /flushdns From: Dave Anderson KG4YZY Sent: Mon 1/14/2008 7:13 PM To: 'Discussion of weather data quality issues' Subject: [wxqc] How rotate.aprs.net is supposed to work.. If you don't care about networking.. or now load balancing the network for CWOP works, then this is your warning that this will be technical in nature, and not weather related, but it does affect the program, so I'm posting it. J If you don't care about this, then just delete this message.. If however you are curious, or work in the IT or ISP industry, here's my thoughts and observations on rotate.aprs.net's functionality. Rotate.aprs.net is nothing more than a DNS record with 4 A records on it for those of you who have experience with DNS. It has a VERY low TTL as well. The primary DNS server that runs APRS is on Findu, so I'm not doing these tests on the same network or server that Findu is on. My local DNS server is the standard run of the mill Bind installation on a SunOS box, no different that the lion share of ISP's run for customers to use. Now, on a different machine on my network, a CentOS5 box, here is what happens if I ping rotate.aprs.net over and over: ldm:~# ping rotate.aprs.net PING rotate.aprs.net (134.173.254.38) 56(84) bytes of data. 64 bytes from SoCal.APRS.Net (134.173.254.38): icmp_seq=0 ttl=54 time=75.5 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 75.527/75.527/75.527/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (67.18.222.130) 56(84) bytes of data. 64 bytes from 82.de.1243.static.theplanet.com (67.18.222.130): icmp_seq=0 ttl=52 time=37.9 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 37.936/37.936/37.936/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (64.58.200.120) 56(84) bytes of data. 64 bytes from fourth.aprsfl.net (64.58.200.120): icmp_seq=0 ttl=128 time=2.19 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 2.196/2.196/2.196/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (165.91.140.28) 56(84) bytes of data. 64 bytes from aprs.tamu.edu (165.91.140.28): icmp_seq=0 ttl=47 time=51.9 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 51.923/51.923/51.923/0.000 ms, pipe 2 ldm:~# ping rotate.aprs.net PING rotate.aprs.net (64.58.200.120) 56(84) bytes of data. 64 bytes from fourth.aprsfl.net (64.58.200.120): icmp_seq=0 ttl=128 time=0.224 ms --- rotate.aprs.net ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms rtt min/avg/max/mdev = 0.224/0.224/0.224/0.000 ms, pipe 2 If you notice every time I did the ping, it did nothing more than a simple gethostname lookup and pinged the IP it resolved. Note it's not in order, it's random, but no matter, every server in 5 pings was hit at least once. No matter if I run this ping on a Linux box or a Windows XP or Vista box, I get the same results every time. What does this mean? Well, many things can mess up DNS caching. First and foremost is a ISP that is running a server overriding the TTL rules we put in place to "save bandwidth" on their network. Not many do this, but we have found a few. Usually users of those ISP's have problems with more than just CWOP though, they'll find times where they can't get to sites say such as CNN.com, because, as you probably guessed it, they also use multiple A records: ldm:~# nslookup www.cnn.com Server: 64.58.200.1 Address: 64.58.200.1#53 Non-authoritative answer: Name: www.cnn.com Address: 64.236.91.21 Name: www.cnn.com Address: 64.236.91.22 Name: www.cnn.com Address: 64.236.91.23 Name: www.cnn.com Address: 64.236.91.24 Name: www.cnn.com Address: 64.236.16.20 Name: www.cnn.com Address: 64.236.16.52 Name: www.cnn.com Address: 64.236.24.12 Name: www.cnn.com Address: 64.236.29.120 Now just as with rotate.aprs.net, every time you go to ping it, you hit a different server. Other issues we've found that can mess with DNS caching are broadband routers with "little built in" DNS servers. My advice is not to use them. Use your ISP's DNS server, unless you run a local copy of bind on Linux or DNS on Windows and talk to the root servers yourself. You'll be much happier with the results. The biggest problem we see to date though is folks who have not updated their CWOP software to the latest version that may correct this. Many of the developers worked overtime to get versions out that do work correctly with round robin DNS, and we thank them for that. There are a few apps that still do not do a fresh lookup every time you call rotate, and those programs will truly get hung on one server till you reload - or in some cases reboot the PC, hopefully those developers of software that does not support this system correctly reading this will realize the severity of this problem and work with us to fix it. There -are- a few who have reported that Windows itself causes this problem, but I have yet to replicate the problem on a service pack 2 XP machine or Vista. Some pre-service pack 2 installs of XP do get "dumb" on the cache, but still only hold the record for an hour if memory serves. At any rate, some folks make DNS sound more complicated than it really is. I've owned and operated a ISP since 1994. Yes, look at your calendar, you'll find that CERN hadn't even released a non beta of the HTTP protocol at that time, it was still gopher and archive for doing lookups online. It isn't for everyone to understand, but overall, it's not that incredibly complicated. Is round robin DNS the ideal and perfect solution? No. I run load balancing boxes using several technologies that work t