Errors when processing weather or stream data - what to do?

Section for FAQs regarding i-Tree Hydro

Moderators: i-Tree Support, i-Tree Team

Post Reply
rcoville
i-Tree Team
Posts: 64
Joined: Fri May 08, 2015 9:29 am
Location: Syracuse, NY

Errors when processing weather or stream data - what to do?

Post by rcoville » Thu Mar 01, 2018 9:49 am

If you are using preloaded data, please write to us as info [at] itreetools.org and we'll help troubleshoot the dataset. If you're using your own datasets obtained or modified outside of i-Tree Hydro, we cannot troubleshoot your specific files for you, but the following information may assist you in troubleshooting on your own.

Here are examples of common errors users face when Hydro isn't able to read in a weather (1st image) or streamflow (2nd image) timeseries:

Image
Weather error example: "Index and length must refer to a location within the string. Parameter name: length"

Image
Streamflow error example: "String was not recognized as a valid DateTime."

First, take a general look at the format of your files vs. the format of a working dataset (e.g. data from the sample project). You can export the sample project's datasets by going to File > Open the Sample Project, then using the options in File > Save Weather and Gage Data. If the format generally looks correct, proceed with more detailed troubleshooting. Otherwise, revisit formatting, consider other reliable data sources, and consider customizing an already-working processed file (also mentioned in the last paragraph of this post) in case that would meet your needs.

These errors indicate that the amount of records (rows) in your files doesn't match up well enough with the amount of records Hydro expects based on your project time period. This can happen due to groups of hours or even entire days of data missing, or too many records for the same hour too many times, or unacceptable format or data types rendering certain rows invalid. It is a tedious process to pinpoint where those errors are and correct them, and we cannot provide that service for you. If you want to troubleshoot on your own, you can use the sample project's datasets for comparison (see instructions on exporting those in the previous paragraph), and the following tips may help.

To begin with, identify which of your files has the shortest time span. Knowing that time span, you can determine how many hours of records should be in your files if there is one record per hour, and then look at how many rows your data has more or less than that. You'll want to ensure your streamflow data and weather data precisely match the time period you enter in Hydro's Step 1. If your data is close to the expected amount of records, you could compare your timestamps to a correct set of timestamps going from start to finish, identifying the locations where your data is offset and different from a correct set of timestamps. For more than minor differences, troubleshoot in this way requires side-by-side comparison with a working time series for the time period your data is meant to represent. That is described more in the next paragraph.

It could be a useful step for you to first obtain a working dataset for the time period you're interested in (for any area - data downloaded directly for stations in the U.S. is more likely to work without modification, though not all NCDC data has sufficient records for Hydro to accept it). For info on obtaining working datasets, see the FAQ on Weather Data or on Streamflow Data. Having a working dataset for your time period of interest gives you timestamps to compare against and troubleshoot your own data with. If your time period of interest is large (especially if it's > 1 year), troubleshooting will be much easier if you break out a smaller time period to work with, at least for your first attempt. Consider that a 365 day year has 8760 hours, which is already a lot of records to troubleshoot - multiple years including leap years will be much more difficult to pinpoint all the errors in.

Lastly, you may want to consider getting a working dataset (again, NCDC data more commonly works for U.S. based locations, and Hydro even has preloaded data for the U.S. between 2005 and 2012) and then customizing it after Hydro V6 processes it, since it is in an easier format to work with. Guidelines on that can be found in the following FAQ, but this is also a process we can't provide specific assistance for: viewtopic.php?f=35&t=1400
A member of the i-Tree Team

Please provide feedback on our technical support services by clicking here to take our 5 minute survey
lalonde340
Posts: 2
Joined: Tue Sep 17, 2019 3:43 pm

Re: Errors when processing weather or stream data - what to do?

Post by lalonde340 » Wed Sep 18, 2019 10:01 am

Hello,
The images on this thread are not visible, can you please re-attach them?
rcoville
i-Tree Team
Posts: 64
Joined: Fri May 08, 2015 9:29 am
Location: Syracuse, NY

Re: Errors when processing weather or stream data - what to do?

Post by rcoville » Tue Apr 21, 2020 5:04 pm

lalonde340 wrote:
Wed Sep 18, 2019 10:01 am
Hello,
The images on this thread are not visible, can you please re-attach them?
Hi lalonde340,

I've updated the original post's image links and have included the error examples as plain text. Please let us know if you still cannot see the error images or have any other feedback.

Thanks and best wishes,
Robbie
A member of the i-Tree Team

Please provide feedback on our technical support services by clicking here to take our 5 minute survey
Post Reply