Top Washington Lakes ratings revisited

[hewesfisher]

Some days I browse things I don't ordinarily look at. Yesterday I happened to go to the home page and for whatever reason I checked out the top rated lakes. I'd like to know what method is used to determine ranking. Something isn't right because there are top rated lakes on the list that shouldn't be.

For example, Wapato Lake in Pierce County is on the list. Wapato has had a total of 14 reports submitted since the inception of WA Lakes. No reports since April 2006, and the single report submitted for that year was a "1". It has been 7 years since Wapato last had a "5" rated report (11/2005). How can Wapato (a juvenile angler only lake) be a top rated lake in WA?

I found similar conditions for several other lakes currently listed as a Top Rated Lake in WA, Columbia Park Pond, another juvenile angler only lake, makes the top 10 with a grand total of 14 reports all time, no reports since 2010, only 1 in 2010, next most current report was 7/06 and it was a "5". I found 2 double posted reports, both rated "5" the first is dated 3/29/03 and the second is 4/10/00 and at least 2 reports for other bodies of water, one for Ringold dated 9/22/01 rated a "5" and one for the Columbia River dated 7/08/06. There are other questionable reports for this 9.8 acre pond due to comments about bass fishing from boats, fishing around an island, etc.

IMO, more than half the lakes in the top rated list shouldn't be there for the same reasons listed above...

Twin Lakes in Ferry County is second highest rated lake in the state. I checked the reports, all 4 of them since 1/1/2010. One report submitted for 2012 and it was a "3". One report submitted for 2011 and it was a "5". Two reports submitted for 2010, one was a "5" and the other a "4". Next most current report was submitted in 2008 and it was a "5". Total reports submitted for this lake - 30.

Conconully Reservoir is #3 on the top lakes list, with a grand total of 5 reports in 2012, One "5", one "4", two "3"s and one "2". That's a 3.4 average for 2012. This lake has had a total of 14 reports since 1/1/2010.

Alta Lake in Okanogan County comes in at #4 with 30 total reports and only 4 reports since June 2006. One report in 2012 a "5", one in 2011 a "2", none in 2010, one in 2009 a "4", none in 2008, one in 2007 a "3". I'd call that a 2.3 average since 2006 yet it is the 4th highest rated lake in WA.

Wanncut in Okanogan County ranks #13 with no reports since 7/2010 and only 19 total reports.

How can some of our most popular lakes like Rufus Woods, Washington, American, Moses, Stevens, Roosevelt, Sammamish, etc. not make the grade? How can a lake with only 30 reports (Twin Lakes) and only one report in 2012, a "3", be the 2nd highest rated lake in the state?

Lake Washington - 176 reports for 2012, 1428 total reports submitted.
Lake Roosevelt - 50 reports for 2012, 457 total reports submitted.
Lake Sammamish - 64 reports for 2012, 508 total reports submitted.
Lake Stevens - 58 reports for 2012, 326 total reports submitted.
American Lake - 119 reports for 2012, 889 total reports submitted.
Rufus Woods - 32 reports for 2012, 480 total reports submitted.
Moses Lake - 18 reports for 2012, 116 total reports submitted.

It appears the methodology used to determine lake ratings is simply the total of lake report scores divided by total number of reports. I confirmed this by running the numbers for Wapato. I think there should be a weighted average, with the most current year having the highest value, each preceding year a lower overall value, and any year with no reports should be a negative value and detract from the overall rating. I'm no statistician, but the current formula really doesn't do justice for rating lakes in WA.

[oneshot]

i think what the angler rates the day as is the determining factor.. or one of them..

i'd like to see a bigger scale.. like top 20 or 30 but based on how many reports as well.

[MotoBoat]

Clearly the equation used to determine the top 10 lakes in Washington, is seriously flawed. Using the report rating is flawed in it's self. The rating given is supposed to be based on catching. But other factors such as, overall experience, wildlife viewed during the trip, weather, fish hooked but not landed is considered to be action, courtesy of others on the water, favorite lake.....ect.......ect!

I too looked at the top 10, and thought to myself....what, why are half those lake on the list, much less high on it? Convinced that made no sense what so ever. I have not looked at it since.

It makes about the same sense. As educating someone that left a fishing report, that their catch contradicts the reg's. And instead of the report being changed, the comments are deleted!. Whaaaaat??????? And Why comes to mind.

Courteous comment, not rude or distasteful ones. But this, unrelated example, used only for comparison should be discussed by starting a new thread.

[Mike Carey]

Yes, Aaron and I agree and our plan is to revamp and improve the way the Top Lakes are determined. It's on his list. I believe he was also looking at changing to a "1-10" range which would make for better variation. I'll alert him to this thread in case he hasn't seen it.

BTW, he is moving the site to our new server tonight, so he'll be busy! The new server will give us a nice jump in site speed, as well as give us a backup server (the old server) meaning (hopefully) no extended down times like we had last month.

[Amx]

It was down last month?

I was lost and had to read the news on the computer.

[MotoBoat]

It was down last month?

I was lost and had to read the news on the computer.

Those moving, muttering emoticon's are fuuuuuny!!!

[Aaron]

I can revisit the Top 10 lakes algorithm if someone wants to throw out some good ideas.

As it stands, and this is just from memory of something I whipped up in 5 minutes years ago, it takes the average of the lakes report ratings, disregards lakes with less than X reports, disregards WHEN the reports were made, and disregards lakes that have a Sponsor.

Come up with something better and more scientific and I'll change it.

MotoBoat: Reports that have "issues" don't usually end when the reporter has been corrected. Unfortunately. Deleting the offending comments is a catch 22. Leave them, and we're usually violating our terms. Delete them, and others that feel the same as the original offenders don't see anyone calling the original poster out, so they do. And it starts again. It goes on and on and on and on and on. Then, after it falls below the radar, people find it again, and it goes on and on and on again. Then the WDFW calls us wanting to talk to the person. Then we politely decline to provide information to them, then, in an effort to just get on with things, we delete it. The person that wrote the report understands by then the error of their ways, and probably correct them. This doesn't do anything for others that might read that report and learn something from it, but I think Mike and I have a level of sanity that we like to maintain, and these troublesome reports push us over the edge and they just have to go. Do you have something different in mind that we could do for reports like this that would allow us to NOT delete them and leave them for others to learn with?

[Mike Carey]

Revamp! Revamp! Revamp!

I vote for adding it to Aaron's never-ending list.

As to the report comments issue. The origination of the fishing report poster having the ability to delete a comment happened because there were guys that would make rude, obnoxious comments to people's reports. Which obviously is a disincentive for anyone to post a fishing report - why would they want to when they get rude comments? So we would get PMs and e-mails about the rude comments and have to deal with each one. Rather than spend a lot of time babysitting over 5,000 reports per year, we decided to let the person posting the report moderate his fishing report's comments. It seemed reasonable - the poster went to the time and effort to post the report, why should some jerk ruin it?

Which brings up your concern about alerting people to rules violations and then having the comment deleted. The solution is to send me a PM regarding the report and I'll deal with it. Which is exactly what happened in this case. However, I will say further, the member that alerted me also send a very nice PM to the person that posted the report and helped him to see that he was indeed, wrong. He in turn contacted me immediately and requested I delete the report, which I did. I really want to thank the person that alerted me publically, but he/she sent it privately so I have to respect their privacy.

So from my point of view, in this case everything worked out just fine. Bottom line is, people can always contact me and alert me to an issue in a report and I'll look into getting it straightened out.

[hewesfisher]

Mike, I'm happy you and Aaron are looking at this, something definitely needs to change. I actually mulled over the thought as to whether a "Top Rated" category is even beneficial or necessary for WA Lakes to have at all.

It's been a lot of years since I took statistics, so I can't be very helpful with developing an algorithm, yet I know there are ways to do weighted averages for periods of time. Read on for my suggestions.

I don't know if changing to a range of 1 - 10 will improve the quality of the overall rating process when people rate an outing a "5" based on a single fish caught. Have seen many reports like that and it's a user input you cannot control. I can also relate a rating scale change to something the USAF did back in the 90s with enlisted performance reports. The scale for eons was 1 - 9 with 1 being the worst and 9 the highest possible. All kinds of criteria was given, only the cream of the crop should attain a 9, blah, blah blah, etc. No matter, over time, the rating system became inflated to the point if you got an 8 it was considered the kiss of death. In the 90s, the rating scale was changed to 1 - 5 with lots of education given to raters (those making the evaluation), only X% of all enlisted folks were expected to get a 5, a 4 was considered an excellent rating, a 3 was supposed to be an average rating, etc. Within a few years, that system fell victim to the same inflated issues of the previous system, and if you got a 4 it was viewed as being a negative report even though it was stated as "Excellent Performer". In the end, changing the rating scale did nothing to resolve the inflation issue or the original system and a report scale change here may follow the same path. There is hope, though, continue to read on.

Things unrelated to "catching" are going to end up in the overall report rating unless each and every report is critically read, reviewed, and user rating adjusted by the admins. Too much work and it will upset a lot of folks.

When my wife and I have a great day on the water, catch some fish, but not a lot, I still have a hard time rating the day less than a "5", but I do. I try hard to be honest and fair with my ratings so they accurately reflect the catching experience, not the outing. There are lots of people who post reports that don't do that, I read them all the time.

I know this is long-winded, but please keep reading. Top rated Lake Chelan had 283 reports submitted for 2012. What percentage was made by us "regular anglers"? Less than 10%, 9.54% to be exact, 27 of the 283 submitted reports. What was the average rating by "regular anglers"? 3.77. I'm not implying anything here, but it's a fact the guide reports skew the overall rating. Guides submit reports for their clients (clients hardly ever submit a report), rarely report less than a 5, although some do rate a 4 once in awhile. There were 5 guide reports rated "3" for 2012, but 10 of 27 from "regular anglers" in 2012 were rated a "3". Is it fair to skew the overall rating in favor of results posted by guides for their clients implying the fishing in Lake Chelan is the best in the state? Not in my opinion, but how does one make a rating system for pros and non-pros fair and equitable?

I think the answer is a paradigm shift away from a rating system based on report score. Why? Because of the inability to be objective. Angler input data is subjective, always will be, and I see no way to rate subjective input objectively. Maybe we should just let the report ratings speak for themselves and do away a rating system that uses the "score" altogether.

I propose a change to a "Most Popular" chart, base it on the number of reports submitted for any given lake over the past 12 months with a declining weight for reports more than a year old, and disregard all reports more than 5yrs old for any body of water. Old reports would still be there for anyone to read, but they wouldn't be counted for popularity.

Here's an example using Lake Chelan:

2012 total reports x 100% 283 x 1 = 283
2011 total reports x 80% 371 x .8 = 297
2010 total reports X 60% 234 x .6 = 140
2009 total reports X 40% 89 x .4 = 36
2008 total reports X 20% 7 x .2 = 1.4

Total popularity score for Lake Chelan would be the sum of the 5 years divided by 5 to get an average, so 757.4 / 5 = 151.48.

Using the same formula for Lake Washington:

2012 176 x 1 = 176
2011 177 x .8 = 142
2010 192 x .6 = 115
2009 145 x .4 = 58
2008 192 x .2 = 38.4

Total popularity score 529.4 / 5 = 105.88

American Lake in Pierce County:

2012 119 x 1 = 119
2011 141 x .8 = 113
2010 169 x .6 = 101
2009 83 x .4 = 33
2008 98 x .2 = 20

American Lake popularity score 386 / 5 = 77.2

My favorite body of water, Lake Roosevelt, Section 18, south end to Spokane Arm:

2012 50 x 1 = 50
2011 94 x .8 = 75
2010 105 x .6 = 63
2009 66 x .4 = 26
2008 27 x .2 = 5

Section 18 popularity, 219 / 5 = 43.8

Twin Lakes in Ferry County (currently # 2 rated lake in WA):

2012 1 x 1 = 1
2011 1 x .8 = .8
2010 2 x .6 = 1.2
2009 0 x .4 = 0
2008 5 x .2 = 1.0

Twin Lakes popularity score 4 / 5 = .8

Using a weighted average, Twin Lakes would not be #2 in the state, nor would it even be on the chart because the weighted average for total reports submitted actually reflects its lack of popularity, just the way it should.

One last observation, the Top Rated feature is currently lakes only, doesn't include rivers. Mike, if you and Aaron think my idea has merit, you could apply the same criteria to the rivers. That way you could have a Top 10 Most Popular Lake AND River chart.

Food for thought.

[Mike Carey]

wow, thanks for the exhaustive analysis. The rating system has always been plagued by the "experience" vrs. "catching" debate. Add to that, guys that fish steelhead or salmon general only get to keep two fish, so if you catch just one fish, it's still a pretty darn good day (at least for this amatuer) whereas if you're trout fishing one fish kinda sucks.

I totally agree something better needs to be implemented. I won't want to get rid of it as I think it is a fun thing for guys to rate their day and also it makes it easy to scan reports if you're looking for those 4 and 5 star days to review.

I guess all I can say is, like many things on the site, it will perculate on our list of priority items to address. I know Aaron has a long list of fixes and wish list items, including the future phone app. If he would just win that darn lottery so he could quit his day job and work fulltime on the web site, all would be good. (maybe I should be wishing that for myself!)

[hewesfisher]

I don't want the report rating to go away either, just don't use it as a the basis for rating the best lakes in the state. That's where it falls apart IMO. Having just become a website/forum owner myself (chief, cook, and bottle washer) I can relate, albeit on a much smaller scale in my case, just how daunting changes and updates can be.

[Mike Carey]

I don't want the report rating to go away either, just don't use it as a the basis for rating the best lakes in the state. That's where it falls apart IMO. Having just become a website/forum owner myself (chief, cook, and bottle washer) I can relate, albeit on a much smaller scale in my case, just how daunting changes and updates can be.

got it, thanks. What's your web site? Wanna plug it here?

[hewesfisher]

I don't want the report rating to go away either, just don't use it as a the basis for rating the best lakes in the state. That's where it falls apart IMO. Having just become a website/forum owner myself (chief, cook, and bottle washer) I can relate, albeit on a much smaller scale in my case, just how daunting changes and updates can be.

got it, thanks. What's your web site? Wanna plug it here?

Sent you a PM.