Continuously sampled eddy-covariance data can be uploaded to the AmeriFlux database using automated processes. Contact the AmeriFlux Data Team to set up this process for your site.
Use these instructions to prepare BASE-In file(s) containing data that are continuously sampled at half-hourly or hourly intervals for a certain period of time (e.g., a month, a year). Data files must have the same time interval between any two sequential values. If you need to upload data reported at other time intervals, contact the AmeriFlux Data Team for help.
Where to review general data variable formats.
These instructions are specific to uploading high frequency data.
- Data processing
- Temporal representativeness and timestamps
- File format and content
- Data variable: Base names
- Data variable: Qualifiers
1. Data processing
Some data processing is necessary before uploading half-hourly / hourly fluxes and meteorological data to the AmeriFlux archive. Follow the guidelines below to make sure your data can be used to generate derived data products by AmeriFlux. Provide data processing information either using the Variable Information Tool (detailed instructions on using this tool), or use the BADM Instrument Ops template in an INSTOM_COMMENT associated with an INSTOM_VARIABLE_H_V_R entry.
1.1 Data quality control
Use your site team’s assessments to apply quality control. This includes removal of data points with bad data (e.g., from sensor failures or applying physical thresholds). Exceptions to this guideline are listed below.
1.2 USTAR filtering
Do not apply USTAR filtering to flux variables. The AmeriFlux Data Team will compute and apply USTAR thresholds. The only exception is for gap-filled data (see below).
1.3 Gap-filling
Non-gap filled data (without USTAR filtering) are required. You can add gap-filled versions of the data too.
Identify gap-filled data using the _F variable qualifier (see Data variable: Qualifiers). Describe the gap-filling method using the Variable Information tool.
2. Temporal representativeness and timestamps
Follow the general instructions described in Data Variables. Keep in mind these following reminders and upload requirements.
- A data file must contain the same time interval throughout the file. Make separate files to upload data reported at different time intervals.
- Use TIMESTAMP_START and TIMESTAMP_END with the YYYYMMDDHHMM format.
Example half-hourly data file:
TIMESTAMP_START,TIMESTAMP_END,CO2,... 201507281700,201507281730,391.1,... 201507281730,201507281800,391.8,... ...
Example hourly data file:
TIMESTAMP_START,TIMESTAMP_END,CO2,... 201507281700,201507281800,391.1,... 201507281800,201507281900,391.8,... ...
- Always put TIMESTAMP_START and TIMESTAMP_END in the first two columns.
- Use local standard time without Daylight Saving Time. Specify time zone using the Site General Information BADM for the site.
- Include data for all days in a leap year.
- Report missing data using -9999 as the replacing value. Other “missing data” values such as -6999, N/A, or NaN may be treated as “real data”—not acceptable!
3. File format and content
3.1 File structure
Format each file to be uploaded as an ASCII* text file using a CSV (comma-separated value) format, i.e., a tabular text format using a comma character to separate values.
Start each submitted file with a row of variable names. No variable name should contain blank spaces. Do not use surrounding quotes. Do not include additional header rows or a row of variable units (see section 4.3 Units).
Use a point as a numeric decimal separator (not a comma). This avoids conflict with commas used for the CSV format.
Example:
TIMESTAMP_START,TIMESTAMP_END,CO2,H2O,FC,... 200210070600,200210070630,375.0026343,13.81902137,2.225711711,... 200210070630,200210070700,375.6178651,13.81904135,1.611090395,... 200210070700,200210070730,375.1484745,13.77998531,1.11762877,... 200210070730,200210070800,374.0334503,13.73454349,0.236125726,... ...
* Note that using an UTF-8 encoding, only the variable labels defined in this document, and numeric values will automatically result in an ASCII file, and thus will be compatible.
3.2 Filename
Format the filename for file uploads as follows (use the csv file extension):
<SITE_ID>_<RESOLUTION>_<TS-START>_<TS-END>_<OPTIONAL>.csv
- <SITE_ID>: Use the AmeriFlux / Fluxnet Site ID in the form CC-AAA. CC is the country code (e.g., US, CA, etc). AAA is the three alphanumeric characters associated with the site. The site ID is determined as part of the site registration process.
- <RESOLUTION>: The time interval used throughout the file. Allowed resolutions are HH (for half-hourly) or HR (for hourly). If you need to upload data at a resolution other than half-hourly or hourly, contact the AmeriFlux Data Team.
- <TS-START>: The timestamp for the file’s earliest data in format of YYYYMMDDHHMM. It is the same as the first entry in the TIMESTAMP_START column.
- <TS-END>: The timestamp of the last data entry in format of YYYYMMDDHHMM. It is the same as the last entry of the TIMESTAMP_END column.
- <OPTIONAL>: A parameter to indicate additional information. This parameter will be removed from the filename upon processing.
3.3 File contents
Timestamps must be continuous. Files can have their first (or last) timestamp be any time, e.g., start (end) mid-year. However, files should not have missing timestamps in the middle of the file. For example, if all variables are missing for an entire week, the data variables should be set to -9999 for the entire week, while the timestamp variables should have valid values.
Data of different time periods can be uploaded using separate files (e.g., months, years). AmeriFlux suggests that data files typically contain at least 3 months or an entire year of data, as well as all variables measured at the site.
If an incomplete data record for the site is uploaded, newly uploaded data may be merged with previous data, if needed, to create a complete site record. For example:
- If the entire data time period previously submitted is re-uploaded, the newly uploaded data will be processed and replace the previously submitted and processed data for that entire time period.
- If the uploaded data contains new data or has only partial overlap with the existing data, the newly uploaded data will be added to the previously submitted and processed data. In the case of overlap, new data will replace existing data.
If you are sending only a portion of your site’s data and do not want the uploaded data merged with previously submitted and processed data, contact the AmeriFlux Data Team.
4. Data variable: Base names
Variables indicate fundamental quantities that are either measured or calculated / derived. They can also indicate quantified quality information.
4.1 Standard variable base names
Use the Data variable: Base names specifications as described in the documentation, with the following exception:
- The only accepted Timekeeping variables are TIMESTAMP_START and TIMESTAMP_END.
4.2 New variable base names
Include measurements not currently described in the standardized list of variables (Data variable: Base names) in the same file containing standard variables. The non-standard variables will be saved for publication in future.
Please also contact us at AmeriFlux Data Team, so that we can start the process of agreeing on variable base names and units and add them to the standardized list.
4.3 Units
Convert data to the units described in the Data variable: Base names description before uploading. The AmeriFlux processing will not do conversions. Notably:
- Please use percentages for ALB, RH, SWC, LEAF_WET and for most variables that could be reported in fractions or percentages.
- Please double-check the units of H2O_SIGMA, CO2_SIGMA, CH4, FCH4, VPD, D_SNOW, WTD.
4.4 Sign conventions
Follow the sign conventions specified in Data variable: Base names.
- For all turbulent flux variables (e.g., gas, heat, momentum flux), a negative value indicates net flux of matter, energy, or momentum from the atmosphere to the ecosystem (flora and fauna). The same convention applies to NEE. GPP and RECO are always positive values, where NEE = RECO – GPP.
- For gas and heat storage fluxes, a positive value indicates net increase of matter or energy storage within the ecosystem.
- For NETRAD, a positive value indicates net energy input from the atmosphere to the ecosystem.
- For RUNOFF, a positive value indicates a net outflow from the ecosystem.
- For G (soil heat flux), a positive value indicates net energy input from the ecosystem to the deep soil.
- For WTD (water table depth), a positive value indicates water level above the ground surface, vice versa.
5. Data variable: Qualifiers
Qualifiers are suffixes appended to variable base names that provide additional information about the variable. Use the same data variable qualifier specifications as described in the Data Variables, with the following modifications:
- The only general qualifier accepted in uploaded data is _F (Gap-filled). Describe the gap-filling method applied in the BADM Instrument Ops template (see Section 1. Data processing).
- Positional qualifiers are accepted as described in the Data variables description.
- Positional qualifiers (e.g., _H_V_R) are generally expected to remain the same at a site. When adding sensors, do not renumber the positional qualifiers without discussing with the AmeriFlux Data Team.
- Aggregation qualifiers are accepted as described in the Data variables description
- Ordering of qualifiers follows the specification described in the Data variables description.