The new manual sets 30 segments as the appropriate sample size for SDAP. This makes for a small sample if you're in a large city. Will FEMA accept it? (I know I can go higher but I need to know if FEMA will accept the 30.)
Thanks.
Will FEMA accept the 30 segment sample size?
Moderators: i-Tree Support, i-Tree Team
-
urban forestry planner
- Posts: 17
- Joined: Tue May 01, 2007 10:31 am
-
StormyJerry
Let me split this up
I am going to try to answer your interesting but complicated question by breaking it into the three different topics that it touches on.
1. As of i-Tree 1.2, 30 is recommended as the maximum sample size, with 10 the minimum. Between those extremes, 2% of the total segment number in the community is recommended.
2. The validity of any sample as an estimation of the entire population generally depends on two factors in statistics: the amount that what you are trying to measure varies from sample unit (for us, segment) to sample unit --known as the variance--and the number of samples.
i-Tree expresses the vailidity of its estimates using the statistical descriptor Standard Error of the Mean (SEM), defined as the standard deviation (a measure of variance) divided by the square root of the sample size.
This means that the more things vary and the smaller the number of sample segments, the greater the margin of error will be. And vice versa. If the rates of pruning/removal and debris are relatively constant across a community, for instance, then a small sample will produce a very good estimate. Even if the estimate ends up having a large margin of error in a particular community, the odds are good that it contains the actual population value.
We make some observations about variance by storm type in Section 3.3.1 of the User's Manual. To our knowledge, this question has not been treated in a rigorous fashion.
3. The reports created by the Storm Damage Assessment Protocol are not sent to FEMA, but to the state emergency office. That office is less interested in the accuracy of any individual estimate than it is in that of the sum of the estimates it receives from the entities reporting. The errors in the various estimates tend to average each other out--some are high, some are low, some are in the middle. The bigger the number of communities using the Protocol, the better the storm-wide estimate will be.
We have presented this Protocol to FEMA officials in the past, and they have been emthusiastic because theycould see that an explicit method developed on a scientific basis was being applied. Most state officials will, we suspect, be equally pleased that the number being submitted is not simply a haphazard guess.
I hope this answers your question. Please let us know you still have concerns.
1. As of i-Tree 1.2, 30 is recommended as the maximum sample size, with 10 the minimum. Between those extremes, 2% of the total segment number in the community is recommended.
2. The validity of any sample as an estimation of the entire population generally depends on two factors in statistics: the amount that what you are trying to measure varies from sample unit (for us, segment) to sample unit --known as the variance--and the number of samples.
i-Tree expresses the vailidity of its estimates using the statistical descriptor Standard Error of the Mean (SEM), defined as the standard deviation (a measure of variance) divided by the square root of the sample size.
This means that the more things vary and the smaller the number of sample segments, the greater the margin of error will be. And vice versa. If the rates of pruning/removal and debris are relatively constant across a community, for instance, then a small sample will produce a very good estimate. Even if the estimate ends up having a large margin of error in a particular community, the odds are good that it contains the actual population value.
We make some observations about variance by storm type in Section 3.3.1 of the User's Manual. To our knowledge, this question has not been treated in a rigorous fashion.
3. The reports created by the Storm Damage Assessment Protocol are not sent to FEMA, but to the state emergency office. That office is less interested in the accuracy of any individual estimate than it is in that of the sum of the estimates it receives from the entities reporting. The errors in the various estimates tend to average each other out--some are high, some are low, some are in the middle. The bigger the number of communities using the Protocol, the better the storm-wide estimate will be.
We have presented this Protocol to FEMA officials in the past, and they have been emthusiastic because theycould see that an explicit method developed on a scientific basis was being applied. Most state officials will, we suspect, be equally pleased that the number being submitted is not simply a haphazard guess.
I hope this answers your question. Please let us know you still have concerns.
30 segment sample size
I am also concerned about the 30 segment sample size.
Is there an example of a city that has used the SDAP pre-storm sampling approach (followed by post storm of course) and recieved reimbursement from FEMA?
As for point three below. We are in a unique urban jurisdiction (District of Columbia) where the "State" Emergency Office exists as a District entity. We would be submitting to ourselves (the District) so we would not benefit from errors in various estimates averaging each other out.
I would love to do on the end of 30, rather than 2% - which would be 267! That would defeat the purpose of rapid assessment.
Robert Corletta
Lead Urban Forester, Urban Forestry Administration
District Department of Transportation
Is there an example of a city that has used the SDAP pre-storm sampling approach (followed by post storm of course) and recieved reimbursement from FEMA?
As for point three below. We are in a unique urban jurisdiction (District of Columbia) where the "State" Emergency Office exists as a District entity. We would be submitting to ourselves (the District) so we would not benefit from errors in various estimates averaging each other out.
I would love to do on the end of 30, rather than 2% - which would be 267! That would defeat the purpose of rapid assessment.
Robert Corletta
Lead Urban Forester, Urban Forestry Administration
District Department of Transportation
It seems to me that this understandable question arises from a confusion betweem an estimate of storm damage and a record of storm damage. Here is what I mean:
In the 24 hours after a storm event, a rapid and accurate estimate is required so that state and then federal disaster declarations can be requested. This is the only moment where SDAP applies. The limit of the 30 samples is a statistically valid estimation technique backed by more than a century of statistical research and use. If questions were ever raised about that basis--and I have trouble imagining how or why that would happen--all that would be required would be to politely suggest the questioner consult a statistician.
The reimbursement from FEMA is by contrast is not based on any estimate but on the actual complete accounting, doccumented by the record kept by an entity, right? Actual dollar amounts have nothing to do with SDAP, because FEMA has its own rigorous criteria for reimbursement that have nothing to do with estimates whatsoever.
I hope this helps.
In the 24 hours after a storm event, a rapid and accurate estimate is required so that state and then federal disaster declarations can be requested. This is the only moment where SDAP applies. The limit of the 30 samples is a statistically valid estimation technique backed by more than a century of statistical research and use. If questions were ever raised about that basis--and I have trouble imagining how or why that would happen--all that would be required would be to politely suggest the questioner consult a statistician.
The reimbursement from FEMA is by contrast is not based on any estimate but on the actual complete accounting, doccumented by the record kept by an entity, right? Actual dollar amounts have nothing to do with SDAP, because FEMA has its own rigorous criteria for reimbursement that have nothing to do with estimates whatsoever.
I hope this helps.
Thought maybe this outline of the disaster process might clarify things a bit:
A Major Disaster Declaration usually follows these steps:
Once FEMA has presidential approval, the process of public assistance kicks in, with which SDAP is not involved.
A Major Disaster Declaration usually follows these steps:
- Local Government responds, supplemented by neighboring communities and volunteer agencies. If overwhelmed, it turns to the State for assistance
- The State responds with its resources, such as the National Guard and state agencies
- Damage Assessment by local, state, federal, and volunteer organizations determines losses and recovery needs [SDAP fits right here!]
- A Major Disaster Declaration is requested by the governor, based on the damage assessment, and an agreement to commit state funds and resources to the long-term ecovery
- FEMA evaluates the request and recommends action to the White House based on the disaster, the local community and the state's ability to recover
- The President approves the request, or FEMA informs the governor it has been denied
Once FEMA has presidential approval, the process of public assistance kicks in, with which SDAP is not involved.
From my personal experience after hurricane Katrina, I found that communities that were showing a willingness to help themselves by having their trees inventoried and damages assessed were more likely to get FEMA assistance than those that did not show a willingness to help themselves. The SDAP is an easy method to show FEMA that your community does have legitimate needs and is willing to help itself in the recovery effort. We at Urban Forestry South are working with the Virginia Urban Forestry State Coordinator and their Emergency Management Agency to develop a post-storm response team of professional arborists to assist communities in the recovery effort. We have incorporated SDAP as our initial response tool. The numbers generated from SDAP allow us to begin a dialogue with FEMA and help us to show them that the communities that we are assisting are committed to helping themselves recover. I hope this helps you in better understanding the use of SDAP.
That's it.
Bingo! Glad our input has helped.
I hope the i-Tree team will review their SDAP documentation with an eye toward making this point clearer for users from the very start.
I hope the i-Tree team will review their SDAP documentation with an eye toward making this point clearer for users from the very start.
-
jerryboone
- Posts: 1
- Joined: Fri Oct 30, 2009 4:20 am