[wxqc] Solar Radiation

Ted Lum gladstonefamily45.net at tedlum.com
Tue Oct 25 09:01:49 CDT 2011


/"Someone mentioned earlier that at some point it was said that there was literally not a single byte left available for storing solar data."/

If you're referring to my earlier comment what I was saying was that I 
was told that solar data was not in the capacity plan. You can't let a 
database run out of space, the application will crash. In fact you may 
need a whole lot of additional space just to accommodate the logs 
depending on your logging strategy and the behavior of your application. 
But in the end you have to come up with a capacity plan that fits your 
disk budget, realizing that you need to provide some extra capacity just 
as a buffer. My understanding, at the time, was that the existing server 
hardware and capacity plan were already stressed and the system was 
running into difficulty keeping up. My understanding, at the time, was 
that there was an unwillingness to add anything to an already stressed 
situation. Again, this is hearsay and its a few years old.

On 10/25/2011 8:28 AM, Rich Mulvey wrote:
> As a software engineer, I'm more than a little familiar with
> suggestions that involve "just add one more field to the database -
> it's easy!" comments, so I'll take a shot at making things a little
> clearer.
>
> First, you keep referring to Findu as an "it".  Findu is not an it -
> it's a "him".  Steve Dimse, in particular.  A guy who has spent -
> what, more than a decade? - of innovating, creating, and maintaining
> Findu.  Continuously.  Without exception.  Think about how many public
> service projects manage to last that long, not even considering the
> scale of Findu.
>
> Findu is not just for weather.  It sucks in all of the APRS reports,
> worldwide.  So it's supporting tens, maybe hundreds of thousands of
> stations constantly transmitting positions, messages, and so on.
> Continuously, 7x365, without a break.
>
> Perhaps that gives you a sense of the scale.
>
> Someone mentioned earlier that at some point it was said that there
> was literally not a single byte left available for storing solar data.
>   I'm not privy to the specifics of the database backend being used,
> but my SWAG based on that single comment is that the database back-end
> being used has maxed out the number of bytes per DB record.
> Considering the length of time Findu has been operating, that's
> entirely possible.
>
> Next, let's make a realistic assessment of the impact of the change.
> Another SWAG of mine is that there are likely to be no more than a
> couple thousand valid solar stations worldwide at the very most.  I'm
> betting it's much less than that, in reality.  So adding solar would
> be a change that supports a miniscule fraction of the stations using
> Findu.
>
> Throwing money at a problem is rarely the solution.  If there are
> backend constraints that preclude adding solar, then "adding one more
> field" turns into "Reengineer the entire system, validate it, and
> migrate it while maintaining DB consistency while at the same time
> keeping it running 24x365 on limited hardware."
>
> You might feel like you're being picked on a bit.  That's because you
> are.  :-)  If you look over the history of the main APRS and Weather
> mailing lists, it's very common for someone to get excited about
> adding new features and start assigning work to the people who
> maintain the system.  Oddly enough, they never seem to follow through,
> regardless of how "easy" and "simple" their demand is.
>
> - Rich
>
>
>
> On Mon, Oct 24, 2011 at 11:10 PM, Matthew McGee<matthewjp-mcgee at att.net>  wrote:
>> I agree.
>>
>>    This is the same point I tryed to make a few days ago. Findu is the middle
>> man. While CWOP and MADIS support solar, Findu doesn't as Ted put it.
>>
>> It cost's alot to expand the servers for solar data. But on one of the
>> forums a fund raiser can be setup for solar support. I would be a happy
>> contributer. I bet many others would be too.
>>
>> I mean Findu spent money on what it supports already. 1 more data
>> field shouldn't make much of a difference if the financial means are
>> provided. If everyone else contributes. If I'm wrong please don't rub it in.
>> But if there's a will there's a way.
>>
>> Ted Lum, How much do you estimate it would cost to add a database line for
>> solar? CWOP doesn't support anything more. Not UV light, not leaf wetness,
>> not when the earth will come to an end LOL.
>>
>>    My point is, Findu supports everything but solar. As CWOP doesn't report
>> anything extra shouldn't be much of issue for one other line if someones
>> helping you financially.
>>
>>    How do you gauge an effort to at least look into going further? Once your
>> almost to the top or finish line you realize you've come along way just to
>> call it quits.
>>
>>   To make that one extra step knowing you don't have to do anything more.
>> Once that's done you can rest knowing you've accomplished the "impossable".
>> just as a note NOTHING is impossable if you try.
>>
>> The only reason I can see Findu doesn't want to support solar is it doesn't
>> want to try. How many request's for solar data have been made besides mine?
>> I can assure Ted it can be done.
>>
>> Ted before saying "NO it's not possable" think it over.
>>
>> ________________________________
>> From: "jdr1 at metdata.com"<jdr1 at metdata.com>
>> To: wxqc at lists.gladstonefamily.net
>> Sent: Mon, October 24, 2011 9:41:09 PM
>> Subject: [wxqc] Solar Radiation
>>
>> I don’t understand why madis  is not collecting this data directly from our
>> stations instead of this third party server.  If they really want and need
>> this data it looks like they would have more interest in making a server
>> available to pull these data sets directly from our stations.
>>
>>
>>
>> JD
>>
>>


-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://server.gladstonefamily.net/pipermail/wxqc/attachments/20111025/d09e9280/attachment.html>


More information about the wxqc mailing list