[wxqc] Comparing stations

Paul Grace paulgrace at lookoutranch.com
Sun Oct 30 11:56:53 CDT 2011


Thanks for the information on this.
If I understand this, if the data is spatially sparse at a location, and
differs considerably, even if it's accurate, it might well be rejected from
the MADIS analysis and the spatial consistency check. (even though the data
is kept, as flagged, it might never be used, we don't know)
 
Wouldn't that allow the MADIS analysis to continue to diverge from reality,
potentially flagging even more data as bad?
Wouldn't it be better to use sparse data to improve the MADIS analysis in
poorly covered areas?
 
Seems like a "lack of data feedback loop"  MADIS' 20F temp error is a pretty
big error it seems to me.  Far bigger than the error from most every
consumer weather station.

  _____  

From: wxqc-bounces at lists.gladstonefamily.net
[mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Ted Lum
Sent: Sunday, October 30, 2011 09:34
To: Discussion of weather data quality issues
Subject: Re: [wxqc] Comparing stations


You have to remember the Philip takes an analysis line from MADIS and makes
his own determination. MADIS just gives out the analysis line, it does not
make any determination. Philip has set some threshold at which point the
software on his site displays that message. You'll often see that he and
MADIS disagree on what is bad data, and both are displayed for that purpose.
If anything it should probably be worded in a less absolute way. But
regardless of any of the quality metrics they are simply metadata, the data
record is kept.

Also, for clarification, I said "..but it does take out flawed readings." It
does not take the data out of the data set, it takes it out of the analysis;
if the data is suspect it won't be used as part of the Spatial Consistency
check. That is bad wording, actually the data won't be removed it will just
not be considered.

On 10/30/2011 12:15 PM, Paul Grace wrote: 

"MADIS does NOT throw out any data at all."  Ah.  I interpreted "Your
readings are not within an acceptable error range" as the data was not
acceptable.  The temp analysis is often wrong for my location by 20F, and
the dew point analysis is similarly horked.  It's humorous-

  _____  

From: wxqc-bounces at lists.gladstonefamily.net
[mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Ted Lum
Sent: Saturday, October 29, 2011 21:27
To: Discussion of weather data quality issues
Subject: Re: [wxqc] Comparing stations


Not flawed in that sense. It checks for stuck sensors. It checks for bounds;
temperature must be between -60 and +130 for example. It checks for Temporal
Consistency; temperature can't vary by more than 35 F/hour for example. It
does an Internal Consistency check; Dew Point Temperature can't exceed Air
Temperature for example. And it does the Spatial Consistency check but we
don't know what station(s) it uses when.

Most of all MADIS does NOT throw out any data at all. Worst it ever does is
flag it with the checks it failed.

On 10/29/2011 10:25 PM, Paul Grace wrote: 

Right.  I think it's not very good at determining what is "flawed" when
looking at data that has no close neighbors.  It exhibits a tendency to
throw out the very data that it most needs to fill in its holes.



From: wxqc-bounces at lists.gladstonefamily.net
[mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Ted Lum
Sent: Saturday, October 29, 2011 19:13
To: Discussion of weather data quality issues
Subject: Re: [wxqc] Comparing stations



Well, we don't really know what MADIS includes so we can't say if it does
that or not, but it does take out flawed readings. Someone at NOAA ESRL/GSD
might be able to say how the algorithm works; I've never seen it as anything
but a black box.

On 10/29/2011 7:34 PM, Paul Grace wrote: 

I would guess that data should be accepted into the model, *unless* it has a
nearby data (hundreds of feet, not hundreds of miles) with which it does not
agree, or exhibits flaws such as stuck, zero, over limit, sensors, etc.



From: wxqc-bounces at lists.gladstonefamily.net
[mailto:wxqc-bounces at lists.gladstonefamily.net] On Behalf Of Ted Lum
Sent: Saturday, October 29, 2011 15:32
To: Discussion of weather data quality issues
Subject: Re: [wxqc] Comparing stations



AFAIK Philip takes the nearest 10 or so. MADIS has its own way of doing
things. I'd put it back on you and ask what would you like it to do? Nothing
can really invent stations and data where there isn't any, nor can it
manufacture a particular type of data. So there is no other data near you,
and no other data that has the same profile as you. What exactly is it that
you want?

On 10/29/2011 4:33 PM, John M. Markle wrote: 

  Not sure what going on here and as always when I ask something on wxforums
I get ingored so I really  don't expect an answer here either.
 Why are station that are 150 miles to 500 miles from me listed on my summa
ry page for comparsion. This is totally useless to me and I ame sure it is
also would be useless to anyone else also. Another thing I notice some of
these 
stations are under Continetal influces where I am under maritime influences.
Seem to be one would want to be compared to station in one own enviroment.
On this list the first station  listed is 150 miles from me and the last one
is 504 miles. Just 140 miles shy of Seattle, Washington.  Other thing I
notice is that they are all PWS stations. Too me this is the poorest choice
to use for comparison as most PWS owner  just throw their station up and
walk away. As long as it seems to be close it is good enough for them.
 
 Here is the link.  http://weather.gladstonefamily.net/site/D1453
 
 
 Like I said I don't expect an answer but thought I try any ways.
 
John 
KL7IFP 







-- 
This message has been scanned for viruses and 
dangerous content by  <http://www.mailscanner.info/> MailScanner, and is 
believed to be clean. 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://server.gladstonefamily.net/pipermail/wxqc/attachments/20111030/d9c28284/attachment-0001.html>


More information about the wxqc mailing list