Customizing timeseries inputs (e.g. weather, observed flow) in Hydro v6

Section for FAQs regarding i-Tree Hydro

Moderators: i-Tree Support, i-Tree Team

Post Reply
rcoville
i-Tree Team
Posts: 64
Joined: Fri May 08, 2015 9:29 am
Location: Syracuse, NY

Customizing timeseries inputs (e.g. weather, observed flow) in Hydro v6

Post by rcoville » Thu Feb 08, 2018 12:05 pm

i-Tree Hydro V6 processes timeseries data - weather, evaporation, and observed streamflow - into a comma delimited format. This is intended to make it easier for users to customize their timeseries inputs.

Comma delimited data can be copy and pasted into spreadsheet software such as Microsoft Excel, and from there the rows of comma delimited data can be separated into columns using a tool such as "Text to Columns" based on the comma delimitations. Data in cells is much easier to modify; a column of hourly precipitation values could be pasted into a processed weather file to replace existing data, which enables advanced international users to overcome the limitation of NOAA weather data not having hourly precipitation outside of the United States.

Step-by-step instructions for customizing precipitation data

Getting processed weather data
  1. Download properly formatted raw weather data from NOAA's Integrated Surface Database (ISD). More detailed instructions for this step are available in the FAQ on weather data in the section about raw weather data: viewtopic.php?f=35&t=1248
  2. Open Hydro V6 and then open the sample project or create or open your own project. In Step 1, input the raw weather data you want to use for your project. Make sure the project time-period matches the time-period of the weather timeseries.
  3. Complete Steps 1 and 2 for your project. Timeseries data processing will occur at the completion of Step 2. Accurately input at least the following inputs for your project, to ensure timeseries inputs are processed accurately:
    • Geographic Reference Location
    • Project Time Period
    • Base Case Tree Canopy %
    • Base Case Canopy Parameters (all LAI and Evergreen % inputs)
  4. After you complete Step 2, you can export the processed timeseries data for customization outside of Hydro. Return to the main Hydro V6 window and save your project if you wish to. To export the processed weather timeseries files, go to File > Save Weather and Gage Data > Save Processed Weather Data, then give a location and name for the processed Weather.dat and Evaporation.dat files which have comma-delimited format.

    Customizing processed weather data
  5. You can now import the processed Weather.dat file into spreadsheet software to customize. In this example, open the Weather.dat file in a text editor (e.g. Notepad++) then copy and paste the entire contents into a new spreadsheet document (e.g. Microsoft Excel 2016).
  6. Use the "Text to Columns" tool (in Excel's Data tab) to separate Weather.dat values into columns. The settings needed to use this tool are that the file is delimited by commas, and the text qualifier is {none}.
  7. Paste in custom precipitation values in the "Precip(m/h)" column. Ensure your units are correct.
  8. Make sure the format of your data in the spreadsheet will match the format of of the processed Weather.dat once it is returned to comma delimited format. Specifically:
    • Copy and paste the cells you wish to export to a Weather.dat file and paste them into a blank, new spreadsheet. This ensures there are not blank data cells at the boundaries of your data set which get carried into the comma delimited file you will export. If this is not done, the file you export may end with extra columns or rows of commas, and extra commas at the end of each row are particularly difficult to manually remove.
    • Make sure the Hr:Min:Sec column is formatted as it is in Weather.dat. To do so, select all the data cells for that column (shortcut: you can click the top data cell, presumably in row 2, then press CTRL+down-arrow to navigate to the last data cell in the column, then hold SHIFT+CTRL as you click the last data cell in the column in order to select all data cells in the column), then right click on the selection and choose "Format Cells...". In the Format Cells window, go to the Numbers tab, go to the Custom category, and enter "hh:mm:ss" as the format type, then press OK. Make sure there are 2 "h" values, as by default there is usually only 1!
    • Get the correct amount of decimal places for the data columns to the right of the "yyyymmdd" and "Hr:Min:Sec" columns. All of those data columns should show 8 decimal places. You can use the shortcut described above (in the point about formatting the Hr:Min:Sec column) to select all relevant cells (presumably C2 to H8737 for a year worth of processed weather data pasted starting at cell A1). In the Home tab, use the Increase Decimal Place (or decrease as needed) to show 8 digits to the right of the decimal point for all of those cells.
    • Make sure precipitation values > 0 are formatting correctly. By default if you format all precip cells as a Number, you may get a warning that those cells with values > 0 are not formatted the same way as neighboring cells - you can click that warning and choose to convert those cells to Numbers, and that generally resolves the issue.
  9. Export your customized data as a .CSV file. In Excel, go to File > Save As > give your file a name and location, and choose the filetype "CSV (Comma delimited) (*.csv)" You'll get a warning about data loss due to using CSV formatting - that is normal and acceptable as columns are collapsed into a row of comma separated values.
  10. In your file browser, navigate to the .CSV file you just saved. Rename the file to end in a .DAT filetype instead of .CSV, as i-Tree Hydro expects .DAT files.

    Using the customized timeseries file in i-Tree Hydro V6
  11. Go back to your Hydro V6 project and into Step 1. For Weather Station Data choose "Select processed weather files" and open the customized weather file .dat you just created and the evaporation.dat associated with it. If this works, you are ready to proceed with project setup steps and run with your custom weather data. If it doesn't work, consider the following troubleshooting steps:
    • If you get an error that your file does not match the project time period, or that your file has > 1 fewer records than expected, then make sure your Project Time Period matches exactly with the time period covered in the timeseries file.
    • If you get an error that your file has exactly 1 fewer or greater records than expected, try the following: close Hydro and open up the customized weather .dat file in a text editor; if there is a blank row under the header row, remove it, and if there is no blank row between the header row and the data timeseries, add a blank row; save the .dat file and retry it in Hydro. This need or a linebreak or not is a bug that should be resolved as we finalize file processing and formatting in Hydro v6 beta.
    • If you continue getting errors, consider removing the last day from your dataset in case timezone conversions in weather processing are dropping the boundary of your timeseries.
    • If you continue getting errors, check to make sure the files you're working with run in Hydro prior to any customization. It may be that the files you're customizing do not work to begin with, in which case you'll need to begin the customization again with a different set of raw or processed weather data.
Related:
A member of the i-Tree Team

Please provide feedback on our technical support services by clicking here to take our 5 minute survey
Post Reply