diff --git a/README.md b/README.md
index 8d86b7e5d7a6932053e0c7653071b5e3e99495d3..ad590ae119ea66ab72fb0b76436d0ed036d2d615 100644
--- a/README.md
+++ b/README.md
@@ -22,8 +22,12 @@ This tool generates relationship information between satellites for the latency
```
#!/home/oper/py39env/bin/python
```
-4. Place the `.htaccess` file in your web root directory to enable CGI script execution
+4. Make sure install Max's sat_latency python library using this and install the requirements.txt
+ ```
+ pip install -r requirements.txt
+ pip install sds-sat-latency --index-url https://gitlab.ssec.wisc.edu/api/v4/projects/2693/packages/pypi/simple
+ ```
## Usage
### Generating Satellite Relationships
@@ -97,10 +101,7 @@ If you're getting "Internal Server Error" in the web interface:
#!/home/oper/py39env/bin/python
```
-4. Review Apache error logs for detailed error messages:
- ```bash
- tail -f /var/log/apache2/error.log
- ```
+
## File Structure
diff --git a/assets/python/latency_viewer.log b/assets/python/latency_viewer.log
deleted file mode 100644
index 2c2ca1eb79021b0dc54c636be26489b53b375c0b..0000000000000000000000000000000000000000
--- a/assets/python/latency_viewer.log
+++ /dev/null
@@ -1,1555 +0,0 @@
-2025-03-18 20:17:15,712 - INFO - ================================================================================
-2025-03-18 20:17:15,712 - INFO - Data.py script starting
-2025-03-18 20:17:15,712 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:15,712 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:15,712 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:15,712 - INFO - User running script: Unknown
-2025-03-18 20:17:15,712 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:15,712 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:15,811 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:15,811 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:15,811 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:15,811 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:17:15,811 - INFO - Getting canonical form for: 4B
-2025-03-18 20:17:15,811 - INFO - Canonical ID: 4B
-2025-03-18 20:17:15,811 - INFO - All variants: ['4B']
-2025-03-18 20:17:15,811 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:17:15,811 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:17:15,811 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:15,811 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:15,811 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:15,811 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:15,811 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:17:16,161 - INFO - Successfully converted data: 2928 records found
-2025-03-18 20:17:16,163 - INFO - Query returned: 2928 records
-2025-03-18 20:17:16,163 - INFO - Converting to DataFrame...
-2025-03-18 20:17:16,167 - INFO - DataFrame created with shape: (2928, 12)
-2025-03-18 20:17:16,167 - INFO - Processing DataFrame...
-2025-03-18 20:17:16,168 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:16,168 - INFO - Cleaning latency data...
-2025-03-18 20:17:16,170 - INFO - DataFrame shape after cleaning: (2928, 12)
-2025-03-18 20:17:16,173 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:16,175 - INFO - Converting timestamps...
-2025-03-18 20:17:16,193 - INFO - Converting to records...
-2025-03-18 20:17:16,221 - INFO - Created 2928 result records
-2025-03-18 20:17:16,223 - INFO - Returning response with status code 200 and 2928 records
-2025-03-18 20:17:22,407 - INFO - ================================================================================
-2025-03-18 20:17:22,407 - INFO - Data.py script starting
-2025-03-18 20:17:22,407 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:22,407 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:22,407 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:22,408 - INFO - User running script: Unknown
-2025-03-18 20:17:22,408 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:22,408 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:22,501 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:22,501 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:22,501 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:22,501 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:17:22,501 - INFO - Getting canonical form for: 4B
-2025-03-18 20:17:22,501 - INFO - Canonical ID: 4B
-2025-03-18 20:17:22,501 - INFO - All variants: ['4B']
-2025-03-18 20:17:22,502 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:17:22,502 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:17:22,502 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:22,502 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:22,502 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:22,502 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:22,502 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:17:22,808 - INFO - Successfully converted data: 2928 records found
-2025-03-18 20:17:22,810 - INFO - Query returned: 2928 records
-2025-03-18 20:17:22,810 - INFO - Converting to DataFrame...
-2025-03-18 20:17:22,814 - INFO - DataFrame created with shape: (2928, 12)
-2025-03-18 20:17:22,814 - INFO - Processing DataFrame...
-2025-03-18 20:17:22,814 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:22,814 - INFO - Cleaning latency data...
-2025-03-18 20:17:22,817 - INFO - DataFrame shape after cleaning: (2928, 12)
-2025-03-18 20:17:22,820 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:22,821 - INFO - Converting timestamps...
-2025-03-18 20:17:22,838 - INFO - Converting to records...
-2025-03-18 20:17:22,864 - INFO - Created 2928 result records
-2025-03-18 20:17:22,865 - INFO - Returning response with status code 200 and 2928 records
-2025-03-18 20:17:29,233 - INFO - ================================================================================
-2025-03-18 20:17:29,233 - INFO - Data.py script starting
-2025-03-18 20:17:29,233 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:29,233 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:29,233 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:29,233 - INFO - User running script: Unknown
-2025-03-18 20:17:29,233 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:29,233 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:29,319 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:29,319 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:29,319 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:29,319 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-18 20:17:29,319 - INFO - Getting canonical form for: G16
-2025-03-18 20:17:29,319 - INFO - Canonical ID: G16
-2025-03-18 20:17:29,319 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:17:29,319 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:17:29,319 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-18 20:17:29,319 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:29,319 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:29,319 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:29,320 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:29,320 - INFO - Filters: satellite_ids=['G16', 'g16'], coverage=None, instrument=['ABI']
-2025-03-18 20:17:34,602 - INFO - Successfully converted data: 277387 records found
-2025-03-18 20:17:34,679 - INFO - Query returned: 277387 records
-2025-03-18 20:17:34,679 - INFO - Converting to DataFrame...
-2025-03-18 20:17:35,070 - INFO - DataFrame created with shape: (277387, 12)
-2025-03-18 20:17:35,070 - INFO - Processing DataFrame...
-2025-03-18 20:17:35,071 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:35,071 - INFO - Cleaning latency data...
-2025-03-18 20:17:35,154 - INFO - DataFrame shape after cleaning: (277387, 12)
-2025-03-18 20:17:35,340 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:35,403 - INFO - Converting timestamps...
-2025-03-18 20:17:35,444 - ERROR - Error during data processing: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:17:35,445 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:17:35,615 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:01:16+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:17:35,615 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:17:42,583 - INFO - ================================================================================
-2025-03-18 20:17:42,583 - INFO - Data.py script starting
-2025-03-18 20:17:42,583 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:42,583 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:42,583 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:42,583 - INFO - User running script: Unknown
-2025-03-18 20:17:42,583 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:42,583 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:42,678 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:42,678 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:42,679 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:42,679 - INFO - Filter parameters: satellite_id=G16, coverage=CONUS, instrument=ABI
-2025-03-18 20:17:42,679 - INFO - Getting canonical form for: G16
-2025-03-18 20:17:42,679 - INFO - Canonical ID: G16
-2025-03-18 20:17:42,679 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:17:42,679 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:17:42,679 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'coverage': 'CONUS', 'instrument': 'ABI'}
-2025-03-18 20:17:42,679 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:42,679 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:42,679 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:42,679 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:42,679 - INFO - Filters: satellite_ids=['G16', 'g16'], coverage=['CONUS'], instrument=['ABI']
-2025-03-18 20:17:43,451 - INFO - Successfully converted data: 25481 records found
-2025-03-18 20:17:43,462 - INFO - Query returned: 25481 records
-2025-03-18 20:17:43,462 - INFO - Converting to DataFrame...
-2025-03-18 20:17:43,500 - INFO - DataFrame created with shape: (25481, 12)
-2025-03-18 20:17:43,501 - INFO - Processing DataFrame...
-2025-03-18 20:17:43,501 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:43,501 - INFO - Cleaning latency data...
-2025-03-18 20:17:43,509 - INFO - DataFrame shape after cleaning: (25481, 12)
-2025-03-18 20:17:43,523 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:43,530 - INFO - Converting timestamps...
-2025-03-18 20:17:43,536 - ERROR - Error during data processing: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 1. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:17:43,537 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 1. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:17:43,549 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:01:16+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 1. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:17:43,550 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:17:47,810 - INFO - ================================================================================
-2025-03-18 20:17:47,810 - INFO - Data.py script starting
-2025-03-18 20:17:47,810 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:47,810 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:47,810 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:47,810 - INFO - User running script: Unknown
-2025-03-18 20:17:47,810 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:47,810 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:47,901 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:47,901 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:47,901 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:47,901 - INFO - Filter parameters: satellite_id=G16, coverage=Full Disk, instrument=ABI
-2025-03-18 20:17:47,901 - INFO - Getting canonical form for: G16
-2025-03-18 20:17:47,902 - INFO - Canonical ID: G16
-2025-03-18 20:17:47,902 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:17:47,902 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:17:47,902 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'coverage': 'Full Disk', 'instrument': 'ABI'}
-2025-03-18 20:17:47,902 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:47,902 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:47,902 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:47,902 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:47,902 - INFO - Filters: satellite_ids=['G16', 'g16'], coverage=['Full Disk'], instrument=['ABI']
-2025-03-18 20:17:48,439 - INFO - Successfully converted data: 13027 records found
-2025-03-18 20:17:48,445 - INFO - Query returned: 13027 records
-2025-03-18 20:17:48,445 - INFO - Converting to DataFrame...
-2025-03-18 20:17:48,461 - INFO - DataFrame created with shape: (13027, 12)
-2025-03-18 20:17:48,461 - INFO - Processing DataFrame...
-2025-03-18 20:17:48,461 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:48,462 - INFO - Cleaning latency data...
-2025-03-18 20:17:48,465 - INFO - DataFrame shape after cleaning: (13027, 12)
-2025-03-18 20:17:48,472 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:48,476 - INFO - Converting timestamps...
-2025-03-18 20:17:48,480 - ERROR - Error during data processing: time data "2025-03-18T00:20:20+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 8. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:17:48,480 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:20:20+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 8. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:17:48,486 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:20:20+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 8. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:17:48,486 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:17:51,246 - INFO - ================================================================================
-2025-03-18 20:17:51,246 - INFO - Data.py script starting
-2025-03-18 20:17:51,246 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:51,246 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:51,246 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:51,246 - INFO - User running script: Unknown
-2025-03-18 20:17:51,246 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:51,246 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:51,341 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:51,341 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:51,342 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:51,342 - INFO - Filter parameters: satellite_id=G18, coverage=Full Disk, instrument=ABI
-2025-03-18 20:17:51,342 - INFO - Getting canonical form for: G18
-2025-03-18 20:17:51,342 - INFO - Canonical ID: G18
-2025-03-18 20:17:51,342 - INFO - All variants: ['G18', 'g18']
-2025-03-18 20:17:51,342 - INFO - Expanded satellite ID G18 to variants: ['G18', 'g18']
-2025-03-18 20:17:51,342 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G18', 'g18'], 'coverage': 'Full Disk', 'instrument': 'ABI'}
-2025-03-18 20:17:51,342 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:51,342 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:51,342 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:51,342 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:51,342 - INFO - Filters: satellite_ids=['G18', 'g18'], coverage=['Full Disk'], instrument=['ABI']
-2025-03-18 20:17:51,927 - INFO - Successfully converted data: 12118 records found
-2025-03-18 20:17:51,933 - INFO - Query returned: 12118 records
-2025-03-18 20:17:51,933 - INFO - Converting to DataFrame...
-2025-03-18 20:17:51,948 - INFO - DataFrame created with shape: (12118, 12)
-2025-03-18 20:17:51,948 - INFO - Processing DataFrame...
-2025-03-18 20:17:51,948 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:17:51,948 - INFO - Cleaning latency data...
-2025-03-18 20:17:51,952 - INFO - DataFrame shape after cleaning: (12118, 12)
-2025-03-18 20:17:51,958 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:17:51,962 - INFO - Converting timestamps...
-2025-03-18 20:17:51,966 - ERROR - Error during data processing: time data "2025-03-18T00:00:20+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 87. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:17:51,966 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:00:20+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 87. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:17:51,972 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:00:20+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 87. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:17:51,972 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:17:59,565 - INFO - ================================================================================
-2025-03-18 20:17:59,565 - INFO - Data.py script starting
-2025-03-18 20:17:59,565 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:17:59,566 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:17:59,566 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:17:59,566 - INFO - User running script: Unknown
-2025-03-18 20:17:59,566 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:17:59,566 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:17:59,661 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:17:59,661 - INFO - Calling data_endpoint function...
-2025-03-18 20:17:59,661 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:17:59,661 - INFO - Filter parameters: satellite_id=G18, coverage=None, instrument=None
-2025-03-18 20:17:59,661 - INFO - Getting canonical form for: G18
-2025-03-18 20:17:59,661 - INFO - Canonical ID: G18
-2025-03-18 20:17:59,661 - INFO - All variants: ['G18', 'g18']
-2025-03-18 20:17:59,661 - INFO - Expanded satellite ID G18 to variants: ['G18', 'g18']
-2025-03-18 20:17:59,661 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G18', 'g18']}
-2025-03-18 20:17:59,661 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:17:59,661 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:17:59,662 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:17:59,662 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:17:59,662 - INFO - Filters: satellite_ids=['g18', 'G18'], coverage=None, instrument=None
-2025-03-18 20:18:05,808 - INFO - Successfully converted data: 305500 records found
-2025-03-18 20:18:05,884 - INFO - Query returned: 305500 records
-2025-03-18 20:18:05,884 - INFO - Converting to DataFrame...
-2025-03-18 20:18:06,300 - INFO - DataFrame created with shape: (305500, 12)
-2025-03-18 20:18:06,300 - INFO - Processing DataFrame...
-2025-03-18 20:18:06,301 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:18:06,301 - INFO - Cleaning latency data...
-2025-03-18 20:18:06,388 - INFO - DataFrame shape after cleaning: (305500, 12)
-2025-03-18 20:18:06,595 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:18:06,664 - INFO - Converting timestamps...
-2025-03-18 20:18:06,705 - ERROR - Error during data processing: time data "2025-03-18T00:00:06.900000+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S%z", at position 1. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:18:06,706 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:00:06.900000+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S%z", at position 1. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:18:06,888 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:00:06.900000+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S%z", at position 1. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:18:06,888 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:18:51,337 - INFO - ================================================================================
-2025-03-18 20:18:51,337 - INFO - Data.py script starting
-2025-03-18 20:18:51,337 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:18:51,337 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:18:51,337 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:18:51,337 - INFO - User running script: Unknown
-2025-03-18 20:18:51,337 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:18:51,337 - INFO - Directory contents: ['data.py', 'metadata.py', 'sat_db_functions.py', 'satellite_relationships.json', 'satellites.py', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log']
-2025-03-18 20:18:51,434 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:18:51,434 - INFO - Calling data_endpoint function...
-2025-03-18 20:18:51,434 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:18:51,434 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:18:51,434 - INFO - Getting canonical form for: 4B
-2025-03-18 20:18:51,434 - INFO - Canonical ID: 4B
-2025-03-18 20:18:51,434 - INFO - All variants: ['4B']
-2025-03-18 20:18:51,435 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:18:51,435 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:18:51,435 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:18:51,435 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:18:51,435 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:18:51,435 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:18:51,435 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:18:51,794 - INFO - Successfully converted data: 2928 records found
-2025-03-18 20:18:51,795 - INFO - Query returned: 2928 records
-2025-03-18 20:18:51,795 - INFO - Converting to DataFrame...
-2025-03-18 20:18:51,800 - INFO - DataFrame created with shape: (2928, 12)
-2025-03-18 20:18:51,800 - INFO - Processing DataFrame...
-2025-03-18 20:18:51,800 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:18:51,800 - INFO - Cleaning latency data...
-2025-03-18 20:18:51,802 - INFO - DataFrame shape after cleaning: (2928, 12)
-2025-03-18 20:18:51,806 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:18:51,808 - INFO - Converting timestamps...
-2025-03-18 20:18:51,826 - INFO - Converting to records...
-2025-03-18 20:18:51,855 - INFO - Created 2928 result records
-2025-03-18 20:18:51,857 - INFO - Returning response with status code 200 and 2928 records
-2025-03-18 20:26:22,802 - INFO - ================================================================================
-2025-03-18 20:26:22,803 - INFO - Data.py script starting
-2025-03-18 20:26:22,803 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:26:22,803 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:26:22,803 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:26:22,803 - INFO - User running script: Unknown
-2025-03-18 20:26:22,803 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:26:22,803 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:26:22,897 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:26:22,897 - INFO - Calling data_endpoint function...
-2025-03-18 20:26:22,897 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:26:22,897 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:26:22,897 - INFO - Getting canonical form for: 4B
-2025-03-18 20:26:22,897 - INFO - Canonical ID: 4B
-2025-03-18 20:26:22,897 - INFO - All variants: ['4B']
-2025-03-18 20:26:22,897 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:26:22,897 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:26:22,897 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:26:22,897 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:26:22,897 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:26:22,897 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:26:22,898 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:26:23,193 - INFO - Successfully converted data: 2955 records found
-2025-03-18 20:26:23,195 - INFO - Query returned: 2955 records
-2025-03-18 20:26:23,195 - INFO - Converting to DataFrame...
-2025-03-18 20:26:23,199 - INFO - DataFrame created with shape: (2955, 12)
-2025-03-18 20:26:23,199 - INFO - Processing DataFrame...
-2025-03-18 20:26:23,200 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:26:23,200 - INFO - Cleaning latency data...
-2025-03-18 20:26:23,202 - INFO - DataFrame shape after cleaning: (2955, 12)
-2025-03-18 20:26:23,206 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:26:23,207 - INFO - Converting timestamps...
-2025-03-18 20:26:23,224 - INFO - Converting to records...
-2025-03-18 20:26:23,251 - INFO - Created 2955 result records
-2025-03-18 20:26:23,252 - INFO - Returning response with status code 200 and 2955 records
-2025-03-18 20:27:13,679 - INFO - ================================================================================
-2025-03-18 20:27:13,679 - INFO - Data.py script starting
-2025-03-18 20:27:13,679 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:27:13,679 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:27:13,679 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:27:13,679 - INFO - User running script: Unknown
-2025-03-18 20:27:13,680 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:27:13,680 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:27:13,769 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:27:13,769 - INFO - Calling data_endpoint function...
-2025-03-18 20:27:13,769 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:27:13,769 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:27:13,769 - INFO - Getting canonical form for: 4B
-2025-03-18 20:27:13,769 - INFO - Canonical ID: 4B
-2025-03-18 20:27:13,769 - INFO - All variants: ['4B']
-2025-03-18 20:27:13,769 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:27:13,769 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:27:13,769 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:27:13,769 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:27:13,769 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:27:13,770 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:27:13,770 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:27:14,055 - INFO - Successfully converted data: 2958 records found
-2025-03-18 20:27:14,057 - INFO - Query returned: 2958 records
-2025-03-18 20:27:14,057 - INFO - Converting to DataFrame...
-2025-03-18 20:27:14,061 - INFO - DataFrame created with shape: (2958, 12)
-2025-03-18 20:27:14,061 - INFO - Processing DataFrame...
-2025-03-18 20:27:14,061 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:27:14,061 - INFO - Cleaning latency data...
-2025-03-18 20:27:14,064 - INFO - DataFrame shape after cleaning: (2958, 12)
-2025-03-18 20:27:14,067 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:27:14,069 - INFO - Converting timestamps...
-2025-03-18 20:27:14,086 - INFO - Converting to records...
-2025-03-18 20:27:14,112 - INFO - Created 2958 result records
-2025-03-18 20:27:14,113 - INFO - Returning response with status code 200 and 2958 records
-2025-03-18 20:28:56,989 - INFO - ================================================================================
-2025-03-18 20:28:56,989 - INFO - Data.py script starting
-2025-03-18 20:28:56,989 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:28:56,989 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:28:56,989 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:28:56,990 - INFO - User running script: Unknown
-2025-03-18 20:28:56,990 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:28:56,990 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:28:57,077 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:28:57,077 - INFO - Calling data_endpoint function...
-2025-03-18 20:28:57,077 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:28:57,077 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:28:57,077 - INFO - Getting canonical form for: 4B
-2025-03-18 20:28:57,077 - INFO - Canonical ID: 4B
-2025-03-18 20:28:57,077 - INFO - All variants: ['4B']
-2025-03-18 20:28:57,077 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:28:57,077 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:28:57,077 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:28:57,077 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:28:57,078 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:28:57,078 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:28:57,078 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:28:57,371 - INFO - Successfully converted data: 2966 records found
-2025-03-18 20:28:57,373 - INFO - Query returned: 2966 records
-2025-03-18 20:28:57,373 - INFO - Converting to DataFrame...
-2025-03-18 20:28:57,377 - INFO - DataFrame created with shape: (2966, 12)
-2025-03-18 20:28:57,377 - INFO - Processing DataFrame...
-2025-03-18 20:28:57,377 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:28:57,377 - INFO - Cleaning latency data...
-2025-03-18 20:28:57,379 - INFO - DataFrame shape after cleaning: (2966, 12)
-2025-03-18 20:28:57,383 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:28:57,384 - INFO - Converting timestamps...
-2025-03-18 20:28:57,401 - INFO - Converting to records...
-2025-03-18 20:28:57,427 - INFO - Created 2966 result records
-2025-03-18 20:28:57,428 - INFO - Returning response with status code 200 and 2966 records
-2025-03-18 20:29:11,045 - INFO - ================================================================================
-2025-03-18 20:29:11,045 - INFO - Data.py script starting
-2025-03-18 20:29:11,045 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:29:11,045 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:29:11,045 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:29:11,045 - INFO - User running script: Unknown
-2025-03-18 20:29:11,045 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:29:11,045 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:29:11,134 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:29:11,134 - INFO - Calling data_endpoint function...
-2025-03-18 20:29:11,135 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:29:11,135 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-18 20:29:11,135 - INFO - Getting canonical form for: G16
-2025-03-18 20:29:11,135 - INFO - Canonical ID: G16
-2025-03-18 20:29:11,135 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:29:11,135 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:29:11,135 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-18 20:29:11,135 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:29:11,135 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:29:11,135 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:29:11,135 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:29:11,135 - INFO - Filters: satellite_ids=['g16', 'G16'], coverage=None, instrument=['ABI']
-2025-03-18 20:29:15,970 - INFO - Successfully converted data: 280045 records found
-2025-03-18 20:29:16,046 - INFO - Query returned: 280045 records
-2025-03-18 20:29:16,046 - INFO - Converting to DataFrame...
-2025-03-18 20:29:16,437 - INFO - DataFrame created with shape: (280045, 12)
-2025-03-18 20:29:16,437 - INFO - Processing DataFrame...
-2025-03-18 20:29:16,438 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:29:16,438 - INFO - Cleaning latency data...
-2025-03-18 20:29:16,520 - INFO - DataFrame shape after cleaning: (280045, 12)
-2025-03-18 20:29:16,701 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:29:16,761 - INFO - Converting timestamps...
-2025-03-18 20:29:16,801 - ERROR - Error during data processing: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-2025-03-18 20:29:16,802 - ERROR - Traceback (most recent call last):
- File "/var/www/html/web_internal/students/ygao/latency/assets/python/data.py", line 198, in data_endpoint
- df['start_time'] = pd.to_datetime(df['start_time']).astype(str)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1063, in to_datetime
- cache_array = _maybe_cache(arg, format, cache, convert_listlike)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 247, in _maybe_cache
- cache_dates = convert_listlike(unique_dates, format)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 433, in _convert_listlike_datetimes
- return _array_strptime_with_fallback(arg, name, utc, format, exact, errors)
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 467, in _array_strptime_with_fallback
- result, tz_out = array_strptime(arg, fmt, exact=exact, errors=errors, utc=utc)
- File "strptime.pyx", line 501, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 451, in pandas._libs.tslibs.strptime.array_strptime
- File "strptime.pyx", line 583, in pandas._libs.tslibs.strptime._parse_with_format
-ValueError: time data "2025-03-18T00:01:16+00:00" doesn't match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:
- - passing `format` if your strings have a consistent format;
- - passing `format='ISO8601'` if your strings are all ISO8601 but not necessarily in exactly the same format;
- - passing `format='mixed'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.
-
-2025-03-18 20:29:16,969 - WARNING - Returning error with status code 500: {'message': 'Data processing error: time data "2025-03-18T00:01:16+00:00" doesn\'t match format "%Y-%m-%dT%H:%M:%S.%f%z", at position 15. You might want to try:\n - passing `format` if your strings have a consistent format;\n - passing `format=\'ISO8601\'` if your strings are all ISO8601 but not necessarily in exactly the same format;\n - passing `format=\'mixed\'`, and the format will be inferred for each element individually. You might want to use `dayfirst` alongside this.', 'data': []}
-2025-03-18 20:29:16,969 - INFO - Returning response with status code 500 and 0 records
-2025-03-18 20:31:58,794 - INFO - ================================================================================
-2025-03-18 20:31:58,794 - INFO - Data.py script starting
-2025-03-18 20:31:58,794 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:31:58,794 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:31:58,794 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:31:58,794 - INFO - User running script: Unknown
-2025-03-18 20:31:58,794 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:31:58,794 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:31:58,885 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:31:58,885 - INFO - Calling data_endpoint function...
-2025-03-18 20:31:58,886 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:31:58,886 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:31:58,886 - INFO - Getting canonical form for: 4B
-2025-03-18 20:31:58,886 - INFO - Canonical ID: 4B
-2025-03-18 20:31:58,886 - INFO - All variants: ['4B']
-2025-03-18 20:31:58,886 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:31:58,886 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:31:58,886 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:31:58,886 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:31:58,886 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:31:58,886 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:31:58,886 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:31:59,181 - INFO - Successfully converted data: 2980 records found
-2025-03-18 20:31:59,182 - INFO - Query returned: 2980 records
-2025-03-18 20:31:59,182 - INFO - Converting to DataFrame...
-2025-03-18 20:31:59,186 - INFO - DataFrame created with shape: (2980, 12)
-2025-03-18 20:31:59,186 - INFO - Processing DataFrame...
-2025-03-18 20:31:59,187 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:31:59,187 - INFO - Cleaning latency data...
-2025-03-18 20:31:59,189 - INFO - DataFrame shape after cleaning: (2980, 12)
-2025-03-18 20:31:59,192 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:31:59,194 - INFO - Converting timestamps...
-2025-03-18 20:31:59,209 - INFO - Converting to records...
-2025-03-18 20:31:59,236 - INFO - Created 2980 result records
-2025-03-18 20:31:59,238 - INFO - Returning response with status code 200 and 2980 records
-2025-03-18 20:32:01,976 - INFO - ================================================================================
-2025-03-18 20:32:01,976 - INFO - Data.py script starting
-2025-03-18 20:32:01,976 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:32:01,976 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:32:01,976 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:32:01,976 - INFO - User running script: Unknown
-2025-03-18 20:32:01,976 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:32:01,976 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:32:02,070 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:32:02,071 - INFO - Calling data_endpoint function...
-2025-03-18 20:32:02,071 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:32:02,071 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:32:02,071 - INFO - Getting canonical form for: 4B
-2025-03-18 20:32:02,071 - INFO - Canonical ID: 4B
-2025-03-18 20:32:02,071 - INFO - All variants: ['4B']
-2025-03-18 20:32:02,071 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:32:02,071 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:32:02,071 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:32:02,071 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:32:02,071 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:32:02,071 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:32:02,071 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:32:02,373 - INFO - Successfully converted data: 2980 records found
-2025-03-18 20:32:02,374 - INFO - Query returned: 2980 records
-2025-03-18 20:32:02,374 - INFO - Converting to DataFrame...
-2025-03-18 20:32:02,378 - INFO - DataFrame created with shape: (2980, 12)
-2025-03-18 20:32:02,378 - INFO - Processing DataFrame...
-2025-03-18 20:32:02,378 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:32:02,378 - INFO - Cleaning latency data...
-2025-03-18 20:32:02,381 - INFO - DataFrame shape after cleaning: (2980, 12)
-2025-03-18 20:32:02,384 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:32:02,386 - INFO - Converting timestamps...
-2025-03-18 20:32:02,400 - INFO - Converting to records...
-2025-03-18 20:32:02,427 - INFO - Created 2980 result records
-2025-03-18 20:32:02,428 - INFO - Returning response with status code 200 and 2980 records
-2025-03-18 20:32:06,230 - INFO - ================================================================================
-2025-03-18 20:32:06,230 - INFO - Data.py script starting
-2025-03-18 20:32:06,230 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:32:06,230 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:32:06,230 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:32:06,230 - INFO - User running script: Unknown
-2025-03-18 20:32:06,230 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:32:06,230 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:32:06,320 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:32:06,320 - INFO - Calling data_endpoint function...
-2025-03-18 20:32:06,321 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:32:06,321 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-18 20:32:06,321 - INFO - Getting canonical form for: G16
-2025-03-18 20:32:06,321 - INFO - Canonical ID: G16
-2025-03-18 20:32:06,321 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:32:06,321 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:32:06,321 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-18 20:32:06,321 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:32:06,321 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:32:06,321 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:32:06,321 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:32:06,321 - INFO - Filters: satellite_ids=['g16', 'G16'], coverage=None, instrument=['ABI']
-2025-03-18 20:32:11,426 - INFO - Successfully converted data: 280815 records found
-2025-03-18 20:32:11,505 - INFO - Query returned: 280815 records
-2025-03-18 20:32:11,505 - INFO - Converting to DataFrame...
-2025-03-18 20:32:11,893 - INFO - DataFrame created with shape: (280815, 12)
-2025-03-18 20:32:11,893 - INFO - Processing DataFrame...
-2025-03-18 20:32:11,893 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:32:11,893 - INFO - Cleaning latency data...
-2025-03-18 20:32:11,978 - INFO - DataFrame shape after cleaning: (280815, 12)
-2025-03-18 20:32:12,162 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:32:12,225 - INFO - Converting timestamps...
-2025-03-18 20:32:13,524 - INFO - Converting to records...
-2025-03-18 20:32:16,223 - INFO - Created 280815 result records
-2025-03-18 20:32:16,397 - INFO - Returning response with status code 200 and 280815 records
-2025-03-18 20:34:55,006 - INFO - ================================================================================
-2025-03-18 20:34:55,006 - INFO - Data.py script starting
-2025-03-18 20:34:55,006 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:34:55,006 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:34:55,006 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:34:55,006 - INFO - User running script: Unknown
-2025-03-18 20:34:55,006 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:34:55,006 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:34:55,093 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:34:55,093 - INFO - Calling data_endpoint function...
-2025-03-18 20:34:55,093 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:34:55,093 - INFO - Filter parameters: satellite_id=g2, coverage=None, instrument=imager
-2025-03-18 20:34:55,093 - INFO - Getting canonical form for: g2
-2025-03-18 20:34:55,093 - INFO - Canonical ID: g2
-2025-03-18 20:34:55,093 - INFO - All variants: ['g2']
-2025-03-18 20:34:55,093 - INFO - Expanded satellite ID g2 to variants: ['g2']
-2025-03-18 20:34:55,093 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['g2'], 'instrument': 'imager'}
-2025-03-18 20:34:55,093 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:34:55,093 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:34:55,093 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:34:55,093 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:34:55,093 - INFO - Filters: satellite_ids=['g2'], coverage=None, instrument=['imager']
-2025-03-18 20:34:55,343 - INFO - Successfully converted data: 84 records found
-2025-03-18 20:34:55,343 - INFO - Query returned: 84 records
-2025-03-18 20:34:55,343 - INFO - Converting to DataFrame...
-2025-03-18 20:34:55,344 - INFO - DataFrame created with shape: (84, 12)
-2025-03-18 20:34:55,344 - INFO - Processing DataFrame...
-2025-03-18 20:34:55,344 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:34:55,344 - INFO - Cleaning latency data...
-2025-03-18 20:34:55,347 - INFO - DataFrame shape after cleaning: (84, 12)
-2025-03-18 20:34:55,349 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:34:55,350 - INFO - Converting timestamps...
-2025-03-18 20:34:55,351 - INFO - Converting to records...
-2025-03-18 20:34:55,355 - INFO - Created 84 result records
-2025-03-18 20:34:55,356 - INFO - Returning response with status code 200 and 84 records
-2025-03-18 20:35:01,533 - INFO - ================================================================================
-2025-03-18 20:35:01,533 - INFO - Data.py script starting
-2025-03-18 20:35:01,533 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:35:01,533 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:35:01,533 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:35:01,533 - INFO - User running script: Unknown
-2025-03-18 20:35:01,533 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:35:01,533 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:35:01,627 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:35:01,627 - INFO - Calling data_endpoint function...
-2025-03-18 20:35:01,627 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:35:01,627 - INFO - Filter parameters: satellite_id=NOAA-18, coverage=None, instrument=amsu
-2025-03-18 20:35:01,627 - INFO - Getting canonical form for: NOAA-18
-2025-03-18 20:35:01,628 - INFO - Canonical ID: NOAA-18
-2025-03-18 20:35:01,628 - INFO - All variants: ['NOAA-18']
-2025-03-18 20:35:01,628 - INFO - Expanded satellite ID NOAA-18 to variants: ['NOAA-18']
-2025-03-18 20:35:01,628 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['NOAA-18'], 'instrument': 'amsu'}
-2025-03-18 20:35:01,628 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:35:01,628 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:35:01,628 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:35:01,628 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:35:01,628 - INFO - Filters: satellite_ids=['NOAA-18'], coverage=None, instrument=['amsu']
-2025-03-18 20:35:01,872 - INFO - Successfully converted data: 9 records found
-2025-03-18 20:35:01,872 - INFO - Query returned: 9 records
-2025-03-18 20:35:01,872 - INFO - Converting to DataFrame...
-2025-03-18 20:35:01,873 - INFO - DataFrame created with shape: (9, 12)
-2025-03-18 20:35:01,873 - INFO - Processing DataFrame...
-2025-03-18 20:35:01,873 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:35:01,873 - INFO - Cleaning latency data...
-2025-03-18 20:35:01,875 - INFO - DataFrame shape after cleaning: (9, 12)
-2025-03-18 20:35:01,877 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:35:01,878 - INFO - Converting timestamps...
-2025-03-18 20:35:01,880 - INFO - Converting to records...
-2025-03-18 20:35:01,883 - INFO - Created 9 result records
-2025-03-18 20:35:01,884 - INFO - Returning response with status code 200 and 9 records
-2025-03-18 20:35:05,542 - INFO - ================================================================================
-2025-03-18 20:35:05,543 - INFO - Data.py script starting
-2025-03-18 20:35:05,543 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:35:05,543 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:35:05,543 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:35:05,543 - INFO - User running script: Unknown
-2025-03-18 20:35:05,543 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:35:05,543 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:35:05,632 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:35:05,632 - INFO - Calling data_endpoint function...
-2025-03-18 20:35:05,633 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:35:05,633 - INFO - Filter parameters: satellite_id=S3B, coverage=None, instrument=SLSTR
-2025-03-18 20:35:05,633 - INFO - Getting canonical form for: S3B
-2025-03-18 20:35:05,633 - INFO - Canonical ID: S3B
-2025-03-18 20:35:05,633 - INFO - All variants: ['S3B']
-2025-03-18 20:35:05,633 - INFO - Expanded satellite ID S3B to variants: ['S3B']
-2025-03-18 20:35:05,633 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['S3B'], 'instrument': 'SLSTR'}
-2025-03-18 20:35:05,633 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:35:05,633 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:35:05,633 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:35:05,633 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:35:05,633 - INFO - Filters: satellite_ids=['S3B'], coverage=None, instrument=['SLSTR']
-2025-03-18 20:35:05,876 - INFO - Successfully converted data: 91 records found
-2025-03-18 20:35:05,876 - INFO - Query returned: 91 records
-2025-03-18 20:35:05,876 - INFO - Converting to DataFrame...
-2025-03-18 20:35:05,877 - INFO - DataFrame created with shape: (91, 12)
-2025-03-18 20:35:05,877 - INFO - Processing DataFrame...
-2025-03-18 20:35:05,877 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:35:05,877 - INFO - Cleaning latency data...
-2025-03-18 20:35:05,880 - INFO - DataFrame shape after cleaning: (91, 12)
-2025-03-18 20:35:05,882 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:35:05,883 - INFO - Converting timestamps...
-2025-03-18 20:35:05,884 - INFO - Converting to records...
-2025-03-18 20:35:05,888 - INFO - Created 91 result records
-2025-03-18 20:35:05,888 - INFO - Returning response with status code 200 and 91 records
-2025-03-18 20:35:10,360 - INFO - ================================================================================
-2025-03-18 20:35:10,360 - INFO - Data.py script starting
-2025-03-18 20:35:10,360 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:35:10,360 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:35:10,360 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:35:10,360 - INFO - User running script: Unknown
-2025-03-18 20:35:10,360 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:35:10,360 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:35:10,454 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:35:10,454 - INFO - Calling data_endpoint function...
-2025-03-18 20:35:10,454 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:35:10,454 - INFO - Filter parameters: satellite_id=snpp, coverage=None, instrument=ATMS
-2025-03-18 20:35:10,454 - INFO - Getting canonical form for: snpp
-2025-03-18 20:35:10,454 - INFO - Canonical ID: snpp
-2025-03-18 20:35:10,454 - INFO - All variants: ['snpp']
-2025-03-18 20:35:10,454 - INFO - Expanded satellite ID snpp to variants: ['snpp']
-2025-03-18 20:35:10,454 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['snpp'], 'instrument': 'ATMS'}
-2025-03-18 20:35:10,454 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:35:10,454 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:35:10,454 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:35:10,455 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:35:10,455 - INFO - Filters: satellite_ids=['snpp'], coverage=None, instrument=['ATMS']
-2025-03-18 20:35:10,652 - INFO - Successfully converted data: 734 records found
-2025-03-18 20:35:10,653 - INFO - Query returned: 734 records
-2025-03-18 20:35:10,653 - INFO - Converting to DataFrame...
-2025-03-18 20:35:10,654 - INFO - DataFrame created with shape: (734, 12)
-2025-03-18 20:35:10,654 - INFO - Processing DataFrame...
-2025-03-18 20:35:10,655 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:35:10,655 - INFO - Cleaning latency data...
-2025-03-18 20:35:10,657 - INFO - DataFrame shape after cleaning: (734, 12)
-2025-03-18 20:35:10,659 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:35:10,660 - INFO - Converting timestamps...
-2025-03-18 20:35:10,665 - INFO - Converting to records...
-2025-03-18 20:35:10,674 - INFO - Created 734 result records
-2025-03-18 20:35:10,674 - INFO - Returning response with status code 200 and 734 records
-2025-03-18 20:35:26,917 - INFO - ================================================================================
-2025-03-18 20:35:26,917 - INFO - Data.py script starting
-2025-03-18 20:35:26,918 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:35:26,918 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:35:26,918 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:35:26,918 - INFO - User running script: Unknown
-2025-03-18 20:35:26,918 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:35:26,918 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:35:27,009 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:35:27,009 - INFO - Calling data_endpoint function...
-2025-03-18 20:35:27,009 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:35:27,009 - INFO - Filter parameters: satellite_id=M10, coverage=None, instrument=seviri
-2025-03-18 20:35:27,009 - INFO - Getting canonical form for: M10
-2025-03-18 20:35:27,009 - INFO - Canonical ID: M10
-2025-03-18 20:35:27,009 - INFO - All variants: ['M10']
-2025-03-18 20:35:27,009 - INFO - Expanded satellite ID M10 to variants: ['M10']
-2025-03-18 20:35:27,009 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['M10'], 'instrument': 'seviri'}
-2025-03-18 20:35:27,009 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:35:27,009 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:35:27,009 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:35:27,009 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:35:27,009 - INFO - Filters: satellite_ids=['M10'], coverage=None, instrument=['seviri']
-2025-03-18 20:35:27,538 - INFO - Successfully converted data: 18528 records found
-2025-03-18 20:35:27,543 - INFO - Query returned: 18528 records
-2025-03-18 20:35:27,543 - INFO - Converting to DataFrame...
-2025-03-18 20:35:27,564 - INFO - DataFrame created with shape: (18528, 12)
-2025-03-18 20:35:27,564 - INFO - Processing DataFrame...
-2025-03-18 20:35:27,564 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:35:27,564 - INFO - Cleaning latency data...
-2025-03-18 20:35:27,568 - INFO - DataFrame shape after cleaning: (18528, 12)
-2025-03-18 20:35:27,577 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:35:27,581 - INFO - Converting timestamps...
-2025-03-18 20:35:27,680 - INFO - Converting to records...
-2025-03-18 20:35:27,833 - INFO - Created 18528 result records
-2025-03-18 20:35:27,840 - INFO - Returning response with status code 200 and 18528 records
-2025-03-18 20:35:32,910 - INFO - ================================================================================
-2025-03-18 20:35:32,910 - INFO - Data.py script starting
-2025-03-18 20:35:32,910 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:35:32,910 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:35:32,910 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:35:32,910 - INFO - User running script: Unknown
-2025-03-18 20:35:32,910 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:35:32,910 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:35:33,003 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:35:33,003 - INFO - Calling data_endpoint function...
-2025-03-18 20:35:33,003 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:35:33,003 - INFO - Filter parameters: satellite_id=S3A, coverage=None, instrument=SLSTR
-2025-03-18 20:35:33,003 - INFO - Getting canonical form for: S3A
-2025-03-18 20:35:33,003 - INFO - Canonical ID: S3A
-2025-03-18 20:35:33,003 - INFO - All variants: ['S3A']
-2025-03-18 20:35:33,003 - INFO - Expanded satellite ID S3A to variants: ['S3A']
-2025-03-18 20:35:33,003 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['S3A'], 'instrument': 'SLSTR'}
-2025-03-18 20:35:33,003 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:35:33,003 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:35:33,003 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:35:33,003 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:35:33,003 - INFO - Filters: satellite_ids=['S3A'], coverage=None, instrument=['SLSTR']
-2025-03-18 20:35:33,274 - INFO - Successfully converted data: 351 records found
-2025-03-18 20:35:33,275 - INFO - Query returned: 351 records
-2025-03-18 20:35:33,275 - INFO - Converting to DataFrame...
-2025-03-18 20:35:33,276 - INFO - DataFrame created with shape: (351, 12)
-2025-03-18 20:35:33,276 - INFO - Processing DataFrame...
-2025-03-18 20:35:33,276 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:35:33,276 - INFO - Cleaning latency data...
-2025-03-18 20:35:33,278 - INFO - DataFrame shape after cleaning: (351, 12)
-2025-03-18 20:35:33,280 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:35:33,281 - INFO - Converting timestamps...
-2025-03-18 20:35:33,284 - INFO - Converting to records...
-2025-03-18 20:35:33,290 - INFO - Created 351 result records
-2025-03-18 20:35:33,290 - INFO - Returning response with status code 200 and 351 records
-2025-03-18 20:38:54,589 - INFO - ================================================================================
-2025-03-18 20:38:54,589 - INFO - Data.py script starting
-2025-03-18 20:38:54,589 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:38:54,589 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:38:54,589 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:38:54,589 - INFO - User running script: Unknown
-2025-03-18 20:38:54,589 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:38:54,589 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:38:54,675 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:38:54,675 - INFO - Calling data_endpoint function...
-2025-03-18 20:38:54,675 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:38:54,675 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:38:54,676 - INFO - Getting canonical form for: 4B
-2025-03-18 20:38:54,676 - INFO - Canonical ID: 4B
-2025-03-18 20:38:54,676 - INFO - All variants: ['4B']
-2025-03-18 20:38:54,676 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:38:54,676 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:38:54,676 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:38:54,676 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:38:54,676 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:38:54,676 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:38:54,676 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:38:54,959 - INFO - Successfully converted data: 2996 records found
-2025-03-18 20:38:54,960 - INFO - Query returned: 2996 records
-2025-03-18 20:38:54,960 - INFO - Converting to DataFrame...
-2025-03-18 20:38:54,964 - INFO - DataFrame created with shape: (2996, 12)
-2025-03-18 20:38:54,964 - INFO - Processing DataFrame...
-2025-03-18 20:38:54,964 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:38:54,964 - INFO - Cleaning latency data...
-2025-03-18 20:38:54,967 - INFO - DataFrame shape after cleaning: (2996, 12)
-2025-03-18 20:38:54,970 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:38:54,971 - INFO - Converting timestamps...
-2025-03-18 20:38:54,987 - INFO - Converting to records...
-2025-03-18 20:38:55,014 - INFO - Created 2996 result records
-2025-03-18 20:38:55,015 - INFO - Returning response with status code 200 and 2996 records
-2025-03-18 20:39:00,749 - INFO - ================================================================================
-2025-03-18 20:39:00,750 - INFO - Data.py script starting
-2025-03-18 20:39:00,750 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:39:00,750 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:39:00,750 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:39:00,750 - INFO - User running script: Unknown
-2025-03-18 20:39:00,750 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:39:00,750 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:39:00,836 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:39:00,837 - INFO - Calling data_endpoint function...
-2025-03-18 20:39:00,837 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-18 20:39:00,837 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:39:00,837 - INFO - Getting canonical form for: 4B
-2025-03-18 20:39:00,837 - INFO - Canonical ID: 4B
-2025-03-18 20:39:00,837 - INFO - All variants: ['4B']
-2025-03-18 20:39:00,837 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:39:00,837 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:39:00,837 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:39:00,837 - INFO - Querying satellite latency data from 2025-03-12T00:00:00 to 2025-03-12T23:59:59
-2025-03-18 20:39:00,837 - INFO - Converted timestamps: 2025-03-12 00:00:00+00:00 to 2025-03-12 23:59:59+00:00
-2025-03-18 20:39:00,837 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-12 00:00:00+00:00, end_date=2025-03-12 23:59:59+00:00
-2025-03-18 20:39:00,838 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:39:02,615 - INFO - Successfully converted data: 3523 records found
-2025-03-18 20:39:02,616 - INFO - Query returned: 3523 records
-2025-03-18 20:39:02,616 - INFO - Converting to DataFrame...
-2025-03-18 20:39:02,621 - INFO - DataFrame created with shape: (3523, 12)
-2025-03-18 20:39:02,621 - INFO - Processing DataFrame...
-2025-03-18 20:39:02,621 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:39:02,621 - INFO - Cleaning latency data...
-2025-03-18 20:39:02,623 - INFO - DataFrame shape after cleaning: (3523, 12)
-2025-03-18 20:39:02,627 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:39:02,628 - INFO - Converting timestamps...
-2025-03-18 20:39:02,646 - INFO - Converting to records...
-2025-03-18 20:39:02,677 - INFO - Created 3523 result records
-2025-03-18 20:39:02,678 - INFO - Returning response with status code 200 and 3523 records
-2025-03-18 20:39:03,395 - INFO - ================================================================================
-2025-03-18 20:39:03,395 - INFO - Data.py script starting
-2025-03-18 20:39:03,395 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:39:03,395 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:39:03,395 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:39:03,396 - INFO - User running script: Unknown
-2025-03-18 20:39:03,396 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:39:03,396 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:39:03,481 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:39:03,481 - INFO - Calling data_endpoint function...
-2025-03-18 20:39:03,482 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-18 20:39:03,482 - INFO - Filter parameters: satellite_id=MetOp-1, coverage=None, instrument=GIIRS
-2025-03-18 20:39:03,482 - INFO - Getting canonical form for: MetOp-1
-2025-03-18 20:39:03,482 - INFO - Canonical ID: MetOp-1
-2025-03-18 20:39:03,482 - INFO - All variants: ['MetOp-1']
-2025-03-18 20:39:03,482 - INFO - Expanded satellite ID MetOp-1 to variants: ['MetOp-1']
-2025-03-18 20:39:03,482 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['MetOp-1'], 'instrument': 'GIIRS'}
-2025-03-18 20:39:03,482 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:39:03,482 - INFO - Querying satellite latency data from 2025-03-12T00:00:00 to 2025-03-12T23:59:59
-2025-03-18 20:39:03,482 - INFO - Converted timestamps: 2025-03-12 00:00:00+00:00 to 2025-03-12 23:59:59+00:00
-2025-03-18 20:39:03,482 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-12 00:00:00+00:00, end_date=2025-03-12 23:59:59+00:00
-2025-03-18 20:39:03,482 - INFO - Filters: satellite_ids=['MetOp-1'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:39:03,628 - INFO - Successfully converted data: 0 records found
-2025-03-18 20:39:03,628 - INFO - Query returned: 0 records
-2025-03-18 20:39:03,628 - INFO - Query returned no data
-2025-03-18 20:39:03,628 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 20:39:04,870 - INFO - ================================================================================
-2025-03-18 20:39:04,870 - INFO - Data.py script starting
-2025-03-18 20:39:04,870 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:39:04,870 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:39:04,870 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:39:04,870 - INFO - User running script: Unknown
-2025-03-18 20:39:04,870 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:39:04,870 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:39:04,956 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:39:04,956 - INFO - Calling data_endpoint function...
-2025-03-18 20:39:04,956 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-18 20:39:04,957 - INFO - Filter parameters: satellite_id=MetOp-1, coverage=None, instrument=GIIRS
-2025-03-18 20:39:04,957 - INFO - Getting canonical form for: MetOp-1
-2025-03-18 20:39:04,957 - INFO - Canonical ID: MetOp-1
-2025-03-18 20:39:04,957 - INFO - All variants: ['MetOp-1']
-2025-03-18 20:39:04,957 - INFO - Expanded satellite ID MetOp-1 to variants: ['MetOp-1']
-2025-03-18 20:39:04,957 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['MetOp-1'], 'instrument': 'GIIRS'}
-2025-03-18 20:39:04,957 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:39:04,957 - INFO - Querying satellite latency data from 2025-03-12T00:00:00 to 2025-03-12T23:59:59
-2025-03-18 20:39:04,957 - INFO - Converted timestamps: 2025-03-12 00:00:00+00:00 to 2025-03-12 23:59:59+00:00
-2025-03-18 20:39:04,957 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-12 00:00:00+00:00, end_date=2025-03-12 23:59:59+00:00
-2025-03-18 20:39:04,957 - INFO - Filters: satellite_ids=['MetOp-1'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:39:05,084 - INFO - Successfully converted data: 0 records found
-2025-03-18 20:39:05,084 - INFO - Query returned: 0 records
-2025-03-18 20:39:05,084 - INFO - Query returned no data
-2025-03-18 20:39:05,084 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 20:39:07,907 - INFO - ================================================================================
-2025-03-18 20:39:07,907 - INFO - Data.py script starting
-2025-03-18 20:39:07,907 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:39:07,908 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:39:07,908 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:39:07,908 - INFO - User running script: Unknown
-2025-03-18 20:39:07,908 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:39:07,908 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:39:07,998 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:39:07,998 - INFO - Calling data_endpoint function...
-2025-03-18 20:39:07,998 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-18 20:39:07,998 - INFO - Filter parameters: satellite_id=MetOp-3, coverage=None, instrument=amsu
-2025-03-18 20:39:07,998 - INFO - Getting canonical form for: MetOp-3
-2025-03-18 20:39:07,998 - INFO - Canonical ID: MetOp-3
-2025-03-18 20:39:07,998 - INFO - All variants: ['MetOp-3']
-2025-03-18 20:39:07,998 - INFO - Expanded satellite ID MetOp-3 to variants: ['MetOp-3']
-2025-03-18 20:39:07,998 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['MetOp-3'], 'instrument': 'amsu'}
-2025-03-18 20:39:07,998 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:39:07,998 - INFO - Querying satellite latency data from 2025-03-12T00:00:00 to 2025-03-12T23:59:59
-2025-03-18 20:39:07,998 - INFO - Converted timestamps: 2025-03-12 00:00:00+00:00 to 2025-03-12 23:59:59+00:00
-2025-03-18 20:39:07,998 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-12 00:00:00+00:00, end_date=2025-03-12 23:59:59+00:00
-2025-03-18 20:39:07,999 - INFO - Filters: satellite_ids=['MetOp-3'], coverage=None, instrument=['amsu']
-2025-03-18 20:39:08,358 - INFO - Successfully converted data: 20 records found
-2025-03-18 20:39:08,358 - INFO - Query returned: 20 records
-2025-03-18 20:39:08,359 - INFO - Converting to DataFrame...
-2025-03-18 20:39:08,359 - INFO - DataFrame created with shape: (20, 12)
-2025-03-18 20:39:08,359 - INFO - Processing DataFrame...
-2025-03-18 20:39:08,360 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:39:08,360 - INFO - Cleaning latency data...
-2025-03-18 20:39:08,362 - INFO - DataFrame shape after cleaning: (20, 12)
-2025-03-18 20:39:08,363 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:39:08,364 - INFO - Converting timestamps...
-2025-03-18 20:39:08,365 - INFO - Converting to records...
-2025-03-18 20:39:08,368 - INFO - Created 20 result records
-2025-03-18 20:39:08,368 - INFO - Returning response with status code 200 and 20 records
-2025-03-18 20:39:15,373 - INFO - ================================================================================
-2025-03-18 20:39:15,373 - INFO - Data.py script starting
-2025-03-18 20:39:15,374 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:39:15,374 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:39:15,374 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:39:15,374 - INFO - User running script: Unknown
-2025-03-18 20:39:15,374 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:39:15,374 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:39:15,460 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:39:15,460 - INFO - Calling data_endpoint function...
-2025-03-18 20:39:15,460 - INFO - Query parameters: start_date=2025-03-26, end_date=2025-03-26, start_hour=00:00, end_hour=23:59
-2025-03-18 20:39:15,460 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:39:15,460 - INFO - Getting canonical form for: 4B
-2025-03-18 20:39:15,460 - INFO - Canonical ID: 4B
-2025-03-18 20:39:15,460 - INFO - All variants: ['4B']
-2025-03-18 20:39:15,460 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:39:15,460 - INFO - Data request - Period: 2025-03-26T00:00:00 to 2025-03-26T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:39:15,461 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:39:15,461 - INFO - Querying satellite latency data from 2025-03-26T00:00:00 to 2025-03-26T23:59:59
-2025-03-18 20:39:15,461 - INFO - Converted timestamps: 2025-03-26 00:00:00+00:00 to 2025-03-26 23:59:59+00:00
-2025-03-18 20:39:15,461 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-26 00:00:00+00:00, end_date=2025-03-26 23:59:59+00:00
-2025-03-18 20:39:15,461 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:39:15,463 - ERROR - Error executing satellite latency query: Must pass schema, or at least one RecordBatch
-2025-03-18 20:39:15,464 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/assets/python/sat_db_functions.py", line 255, in run_sat_latency_query
- data = satellite_data_from_filters(
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/sat_latency/interface.py", line 266, in satellite_data_from_filters
- df = read_satellite_data(
- File "/home/ygao/miniconda/envs/py39env/lib/python3.9/site-packages/sat_latency/pipeline/load.py", line 101, in read_satellite_data
- tbl = pa.Table.from_batches(
- File "pyarrow/table.pxi", line 4987, in pyarrow.lib.Table.from_batches
-ValueError: Must pass schema, or at least one RecordBatch
-
-2025-03-18 20:39:15,464 - INFO - Query returned: 0 records
-2025-03-18 20:39:15,464 - INFO - Query returned no data
-2025-03-18 20:39:15,464 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 20:42:03,119 - INFO - ================================================================================
-2025-03-18 20:42:03,119 - INFO - Data.py script starting
-2025-03-18 20:42:03,120 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:42:03,120 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:42:03,120 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:42:03,120 - INFO - User running script: Unknown
-2025-03-18 20:42:03,120 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:42:03,120 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:42:03,205 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:42:03,205 - INFO - Calling data_endpoint function...
-2025-03-18 20:42:03,206 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:42:03,206 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:42:03,206 - INFO - Getting canonical form for: 4B
-2025-03-18 20:42:03,206 - INFO - Canonical ID: 4B
-2025-03-18 20:42:03,206 - INFO - All variants: ['4B']
-2025-03-18 20:42:03,206 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:42:03,206 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:42:03,206 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:42:03,206 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:42:03,206 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:42:03,206 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:42:03,206 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:42:03,490 - INFO - Successfully converted data: 3008 records found
-2025-03-18 20:42:03,491 - INFO - Query returned: 3008 records
-2025-03-18 20:42:03,491 - INFO - Converting to DataFrame...
-2025-03-18 20:42:03,495 - INFO - DataFrame created with shape: (3008, 12)
-2025-03-18 20:42:03,495 - INFO - Processing DataFrame...
-2025-03-18 20:42:03,495 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:42:03,495 - INFO - Cleaning latency data...
-2025-03-18 20:42:03,497 - INFO - DataFrame shape after cleaning: (3008, 12)
-2025-03-18 20:42:03,501 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:42:03,502 - INFO - Converting timestamps...
-2025-03-18 20:42:03,516 - INFO - Converting to records...
-2025-03-18 20:42:03,541 - INFO - Created 3008 result records
-2025-03-18 20:42:03,542 - INFO - Returning response with status code 200 and 3008 records
-2025-03-18 20:42:08,664 - INFO - ================================================================================
-2025-03-18 20:42:08,664 - INFO - Data.py script starting
-2025-03-18 20:42:08,664 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:42:08,664 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:42:08,664 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:42:08,664 - INFO - User running script: Unknown
-2025-03-18 20:42:08,664 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:42:08,664 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:42:08,752 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:42:08,752 - INFO - Calling data_endpoint function...
-2025-03-18 20:42:08,752 - INFO - Query parameters: start_date=2025-03-17, end_date=2025-03-17, start_hour=00:00, end_hour=23:59
-2025-03-18 20:42:08,752 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 20:42:08,752 - INFO - Getting canonical form for: 4B
-2025-03-18 20:42:08,752 - INFO - Canonical ID: 4B
-2025-03-18 20:42:08,752 - INFO - All variants: ['4B']
-2025-03-18 20:42:08,753 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 20:42:08,753 - INFO - Data request - Period: 2025-03-17T00:00:00 to 2025-03-17T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 20:42:08,753 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:42:08,753 - INFO - Querying satellite latency data from 2025-03-17T00:00:00 to 2025-03-17T23:59:59
-2025-03-18 20:42:08,753 - INFO - Converted timestamps: 2025-03-17 00:00:00+00:00 to 2025-03-17 23:59:59+00:00
-2025-03-18 20:42:08,753 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-17 00:00:00+00:00, end_date=2025-03-17 23:59:59+00:00
-2025-03-18 20:42:08,753 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 20:42:09,103 - INFO - Successfully converted data: 3580 records found
-2025-03-18 20:42:09,105 - INFO - Query returned: 3580 records
-2025-03-18 20:42:09,105 - INFO - Converting to DataFrame...
-2025-03-18 20:42:09,110 - INFO - DataFrame created with shape: (3580, 12)
-2025-03-18 20:42:09,110 - INFO - Processing DataFrame...
-2025-03-18 20:42:09,110 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:42:09,110 - INFO - Cleaning latency data...
-2025-03-18 20:42:09,112 - INFO - DataFrame shape after cleaning: (3580, 12)
-2025-03-18 20:42:09,116 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:42:09,117 - INFO - Converting timestamps...
-2025-03-18 20:42:09,134 - INFO - Converting to records...
-2025-03-18 20:42:09,164 - INFO - Created 3580 result records
-2025-03-18 20:42:09,165 - INFO - Returning response with status code 200 and 3580 records
-2025-03-18 20:43:28,417 - INFO - ================================================================================
-2025-03-18 20:43:28,417 - INFO - Data.py script starting
-2025-03-18 20:43:28,417 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:43:28,417 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:43:28,417 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:43:28,417 - INFO - User running script: Unknown
-2025-03-18 20:43:28,417 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:43:28,417 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:43:28,516 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:43:28,516 - INFO - Calling data_endpoint function...
-2025-03-18 20:43:28,517 - INFO - Query parameters: start_date=2025-03-17, end_date=2025-03-17, start_hour=00:00, end_hour=23:59
-2025-03-18 20:43:28,517 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-18 20:43:28,517 - INFO - Getting canonical form for: G16
-2025-03-18 20:43:28,517 - INFO - Canonical ID: G16
-2025-03-18 20:43:28,517 - INFO - All variants: ['G16', 'g16']
-2025-03-18 20:43:28,517 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-18 20:43:28,517 - INFO - Data request - Period: 2025-03-17T00:00:00 to 2025-03-17T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-18 20:43:28,517 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:43:28,517 - INFO - Querying satellite latency data from 2025-03-17T00:00:00 to 2025-03-17T23:59:59
-2025-03-18 20:43:28,517 - INFO - Converted timestamps: 2025-03-17 00:00:00+00:00 to 2025-03-17 23:59:59+00:00
-2025-03-18 20:43:28,517 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-17 00:00:00+00:00, end_date=2025-03-17 23:59:59+00:00
-2025-03-18 20:43:28,517 - INFO - Filters: satellite_ids=['g16', 'G16'], coverage=None, instrument=['ABI']
-2025-03-18 20:43:35,104 - INFO - Successfully converted data: 331761 records found
-2025-03-18 20:43:35,200 - INFO - Query returned: 331761 records
-2025-03-18 20:43:35,200 - INFO - Converting to DataFrame...
-2025-03-18 20:43:35,688 - INFO - DataFrame created with shape: (331761, 12)
-2025-03-18 20:43:35,688 - INFO - Processing DataFrame...
-2025-03-18 20:43:35,689 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:43:35,689 - INFO - Cleaning latency data...
-2025-03-18 20:43:35,792 - INFO - DataFrame shape after cleaning: (331761, 12)
-2025-03-18 20:43:36,026 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:43:36,103 - INFO - Converting timestamps...
-2025-03-18 20:43:37,821 - INFO - Converting to records...
-2025-03-18 20:43:41,365 - INFO - Created 331761 result records
-2025-03-18 20:43:41,578 - INFO - Returning response with status code 200 and 331761 records
-2025-03-18 20:53:06,558 - INFO - ================================================================================
-2025-03-18 20:53:06,558 - INFO - Data.py script starting
-2025-03-18 20:53:06,558 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 20:53:06,558 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 20:53:06,558 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 20:53:06,559 - INFO - User running script: Unknown
-2025-03-18 20:53:06,559 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 20:53:06,559 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'test.py', 'test_log', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py']
-2025-03-18 20:53:06,659 - INFO - Successfully imported from sat_db_functions
-2025-03-18 20:53:06,659 - INFO - Calling data_endpoint function...
-2025-03-18 20:53:06,660 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 20:53:06,660 - INFO - Filter parameters: satellite_id=M10, coverage=None, instrument=seviri
-2025-03-18 20:53:06,660 - INFO - Getting canonical form for: M10
-2025-03-18 20:53:06,660 - INFO - Canonical ID: M10
-2025-03-18 20:53:06,660 - INFO - All variants: ['M10']
-2025-03-18 20:53:06,660 - INFO - Expanded satellite ID M10 to variants: ['M10']
-2025-03-18 20:53:06,660 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['M10'], 'instrument': 'seviri'}
-2025-03-18 20:53:06,660 - INFO - About to call run_sat_latency_query...
-2025-03-18 20:53:06,660 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 20:53:06,660 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 20:53:06,660 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 20:53:06,660 - INFO - Filters: satellite_ids=['M10'], coverage=None, instrument=['seviri']
-2025-03-18 20:53:07,186 - INFO - Successfully converted data: 19127 records found
-2025-03-18 20:53:07,191 - INFO - Query returned: 19127 records
-2025-03-18 20:53:07,191 - INFO - Converting to DataFrame...
-2025-03-18 20:53:07,216 - INFO - DataFrame created with shape: (19127, 12)
-2025-03-18 20:53:07,216 - INFO - Processing DataFrame...
-2025-03-18 20:53:07,216 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 20:53:07,216 - INFO - Cleaning latency data...
-2025-03-18 20:53:07,221 - INFO - DataFrame shape after cleaning: (19127, 12)
-2025-03-18 20:53:07,231 - INFO - Adding canonical_satellite_id column...
-2025-03-18 20:53:07,236 - INFO - Converting timestamps...
-2025-03-18 20:53:07,341 - INFO - Converting to records...
-2025-03-18 20:53:07,515 - INFO - Created 19127 result records
-2025-03-18 20:53:07,523 - INFO - Returning response with status code 200 and 19127 records
-2025-03-18 21:03:20,482 - INFO - ================================================================================
-2025-03-18 21:03:20,482 - INFO - Data.py script starting
-2025-03-18 21:03:20,483 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:03:20,483 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:03:20,483 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:03:20,483 - INFO - User running script: Unknown
-2025-03-18 21:03:20,483 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:03:20,483 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:03:20,583 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:03:20,583 - INFO - Calling data_endpoint function...
-2025-03-18 21:03:20,583 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:03:20,583 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:03:20,583 - INFO - Getting canonical form for: 4B
-2025-03-18 21:03:20,583 - INFO - Canonical ID: 4B
-2025-03-18 21:03:20,583 - INFO - All variants: ['4B']
-2025-03-18 21:03:20,583 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:03:20,583 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:03:20,583 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:03:20,583 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:03:20,583 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:03:20,583 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:03:20,583 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:03:20,913 - INFO - Successfully converted data: 3038 records found
-2025-03-18 21:03:20,915 - INFO - Query returned: 3038 records
-2025-03-18 21:03:20,915 - INFO - Converting to DataFrame...
-2025-03-18 21:03:20,919 - INFO - DataFrame created with shape: (3038, 12)
-2025-03-18 21:03:20,919 - INFO - Processing DataFrame...
-2025-03-18 21:03:20,919 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:03:20,919 - INFO - Cleaning latency data...
-2025-03-18 21:03:20,922 - INFO - DataFrame shape after cleaning: (3038, 12)
-2025-03-18 21:03:20,925 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:03:20,927 - INFO - Converting timestamps...
-2025-03-18 21:03:20,943 - INFO - Converting to records...
-2025-03-18 21:03:20,972 - INFO - Created 3038 result records
-2025-03-18 21:03:20,973 - INFO - Returning response with status code 200 and 3038 records
-2025-03-18 21:04:08,659 - INFO - ================================================================================
-2025-03-18 21:04:08,659 - INFO - Data.py script starting
-2025-03-18 21:04:08,659 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:04:08,659 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:04:08,659 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:04:08,660 - INFO - User running script: Unknown
-2025-03-18 21:04:08,660 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:04:08,660 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:04:09,069 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:04:09,069 - INFO - Calling data_endpoint function...
-2025-03-18 21:04:09,069 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:04:09,069 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:04:09,069 - INFO - Getting canonical form for: 4B
-2025-03-18 21:04:09,069 - INFO - Canonical ID: 4B
-2025-03-18 21:04:09,070 - INFO - All variants: ['4B']
-2025-03-18 21:04:09,070 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:04:09,070 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:04:09,070 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:04:09,070 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:04:09,070 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:04:09,070 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:04:09,070 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:04:09,384 - INFO - Successfully converted data: 3038 records found
-2025-03-18 21:04:09,386 - INFO - Query returned: 3038 records
-2025-03-18 21:04:09,386 - INFO - Converting to DataFrame...
-2025-03-18 21:04:09,390 - INFO - DataFrame created with shape: (3038, 12)
-2025-03-18 21:04:09,390 - INFO - Processing DataFrame...
-2025-03-18 21:04:09,390 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:04:09,390 - INFO - Cleaning latency data...
-2025-03-18 21:04:09,393 - INFO - DataFrame shape after cleaning: (3038, 12)
-2025-03-18 21:04:09,396 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:04:09,398 - INFO - Converting timestamps...
-2025-03-18 21:04:09,415 - INFO - Converting to records...
-2025-03-18 21:04:09,444 - INFO - Created 3038 result records
-2025-03-18 21:04:09,445 - INFO - Returning response with status code 200 and 3038 records
-2025-03-18 21:25:14,863 - INFO - ================================================================================
-2025-03-18 21:25:14,863 - INFO - Data.py script starting
-2025-03-18 21:25:14,863 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:25:14,863 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:25:14,863 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:25:14,863 - INFO - User running script: Unknown
-2025-03-18 21:25:14,864 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:25:14,864 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:25:15,297 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:25:15,298 - INFO - Calling data_endpoint function...
-2025-03-18 21:25:15,298 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:25:15,298 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:25:15,298 - INFO - Getting canonical form for: 4B
-2025-03-18 21:25:15,298 - INFO - Canonical ID: 4B
-2025-03-18 21:25:15,298 - INFO - All variants: ['4B']
-2025-03-18 21:25:15,298 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:25:15,298 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:25:15,298 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:25:15,298 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:25:15,298 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:25:15,298 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:25:15,298 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:25:15,643 - INFO - Successfully converted data: 3069 records found
-2025-03-18 21:25:15,645 - INFO - Query returned: 3069 records
-2025-03-18 21:25:15,645 - INFO - Converting to DataFrame...
-2025-03-18 21:25:15,649 - INFO - DataFrame created with shape: (3069, 12)
-2025-03-18 21:25:15,649 - INFO - Processing DataFrame...
-2025-03-18 21:25:15,650 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:25:15,650 - INFO - Cleaning latency data...
-2025-03-18 21:25:15,652 - INFO - DataFrame shape after cleaning: (3069, 12)
-2025-03-18 21:25:15,656 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:25:15,657 - INFO - Converting timestamps...
-2025-03-18 21:25:15,672 - INFO - Converting to records...
-2025-03-18 21:25:15,700 - INFO - Created 3069 result records
-2025-03-18 21:25:15,701 - INFO - Returning response with status code 200 and 3069 records
-2025-03-18 21:25:49,489 - INFO - ================================================================================
-2025-03-18 21:25:49,489 - INFO - Data.py script starting
-2025-03-18 21:25:49,489 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:25:49,489 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:25:49,489 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:25:49,489 - INFO - User running script: Unknown
-2025-03-18 21:25:49,489 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:25:49,489 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:25:49,911 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:25:49,911 - INFO - Calling data_endpoint function...
-2025-03-18 21:25:49,911 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:25:49,911 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:25:49,912 - INFO - Getting canonical form for: 4B
-2025-03-18 21:25:49,912 - INFO - Canonical ID: 4B
-2025-03-18 21:25:49,912 - INFO - All variants: ['4B']
-2025-03-18 21:25:49,912 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:25:49,912 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:25:49,912 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:25:49,912 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:25:49,912 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:25:49,912 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:25:49,912 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:25:50,296 - INFO - Successfully converted data: 3072 records found
-2025-03-18 21:25:50,298 - INFO - Query returned: 3072 records
-2025-03-18 21:25:50,298 - INFO - Converting to DataFrame...
-2025-03-18 21:25:50,302 - INFO - DataFrame created with shape: (3072, 12)
-2025-03-18 21:25:50,302 - INFO - Processing DataFrame...
-2025-03-18 21:25:50,303 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:25:50,303 - INFO - Cleaning latency data...
-2025-03-18 21:25:50,305 - INFO - DataFrame shape after cleaning: (3072, 12)
-2025-03-18 21:25:50,309 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:25:50,311 - INFO - Converting timestamps...
-2025-03-18 21:25:50,327 - INFO - Converting to records...
-2025-03-18 21:25:50,357 - INFO - Created 3072 result records
-2025-03-18 21:25:50,359 - INFO - Returning response with status code 200 and 3072 records
-2025-03-18 21:27:39,758 - INFO - ================================================================================
-2025-03-18 21:27:39,758 - INFO - Data.py script starting
-2025-03-18 21:27:39,758 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:27:39,758 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:27:39,758 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:27:39,758 - INFO - User running script: Unknown
-2025-03-18 21:27:39,758 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:27:39,758 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:27:40,206 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:27:40,206 - INFO - Calling data_endpoint function...
-2025-03-18 21:27:40,206 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:27:40,206 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:27:40,206 - INFO - Getting canonical form for: 4B
-2025-03-18 21:27:40,206 - INFO - Canonical ID: 4B
-2025-03-18 21:27:40,206 - INFO - All variants: ['4B']
-2025-03-18 21:27:40,206 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:27:40,206 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:27:40,206 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:27:40,206 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:27:40,206 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:27:40,206 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:27:40,206 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:27:40,601 - INFO - Successfully converted data: 3078 records found
-2025-03-18 21:27:40,602 - INFO - Query returned: 3078 records
-2025-03-18 21:27:40,603 - INFO - Converting to DataFrame...
-2025-03-18 21:27:40,607 - INFO - DataFrame created with shape: (3078, 12)
-2025-03-18 21:27:40,607 - INFO - Processing DataFrame...
-2025-03-18 21:27:40,608 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:27:40,608 - INFO - Cleaning latency data...
-2025-03-18 21:27:40,610 - INFO - DataFrame shape after cleaning: (3078, 12)
-2025-03-18 21:27:40,614 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:27:40,615 - INFO - Converting timestamps...
-2025-03-18 21:27:40,632 - INFO - Converting to records...
-2025-03-18 21:27:40,665 - INFO - Created 3078 result records
-2025-03-18 21:27:40,666 - INFO - Returning response with status code 200 and 3078 records
-2025-03-18 21:28:34,918 - INFO - ================================================================================
-2025-03-18 21:28:34,918 - INFO - Data.py script starting
-2025-03-18 21:28:34,918 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:28:34,918 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:28:34,918 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:28:34,918 - INFO - User running script: Unknown
-2025-03-18 21:28:34,918 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:28:34,918 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:28:35,301 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:28:35,301 - INFO - Calling data_endpoint function...
-2025-03-18 21:28:35,301 - INFO - Query parameters: start_date=2025-03-16, end_date=2025-03-16, start_hour=00:00, end_hour=23:59
-2025-03-18 21:28:35,301 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:28:35,301 - INFO - Getting canonical form for: 4B
-2025-03-18 21:28:35,301 - INFO - Canonical ID: 4B
-2025-03-18 21:28:35,301 - INFO - All variants: ['4B']
-2025-03-18 21:28:35,301 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:28:35,301 - INFO - Data request - Period: 2025-03-16T00:00:00 to 2025-03-16T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:28:35,301 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:28:35,301 - INFO - Querying satellite latency data from 2025-03-16T00:00:00 to 2025-03-16T23:59:59
-2025-03-18 21:28:35,301 - INFO - Converted timestamps: 2025-03-16 00:00:00+00:00 to 2025-03-16 23:59:59+00:00
-2025-03-18 21:28:35,301 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-16 00:00:00+00:00, end_date=2025-03-16 23:59:59+00:00
-2025-03-18 21:28:35,302 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:28:36,752 - INFO - Successfully converted data: 3576 records found
-2025-03-18 21:28:36,754 - INFO - Query returned: 3576 records
-2025-03-18 21:28:36,754 - INFO - Converting to DataFrame...
-2025-03-18 21:28:36,759 - INFO - DataFrame created with shape: (3576, 12)
-2025-03-18 21:28:36,759 - INFO - Processing DataFrame...
-2025-03-18 21:28:36,759 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:28:36,759 - INFO - Cleaning latency data...
-2025-03-18 21:28:36,761 - INFO - DataFrame shape after cleaning: (3576, 12)
-2025-03-18 21:28:36,765 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:28:36,766 - INFO - Converting timestamps...
-2025-03-18 21:28:36,782 - INFO - Converting to records...
-2025-03-18 21:28:36,814 - INFO - Created 3576 result records
-2025-03-18 21:28:36,815 - INFO - Returning response with status code 200 and 3576 records
-2025-03-18 21:33:01,664 - INFO - ================================================================================
-2025-03-18 21:33:01,664 - INFO - Data.py script starting
-2025-03-18 21:33:01,664 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:33:01,664 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:33:01,664 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:33:01,664 - INFO - User running script: Unknown
-2025-03-18 21:33:01,664 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:33:01,664 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:33:02,079 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:33:02,079 - INFO - Calling data_endpoint function...
-2025-03-18 21:33:02,079 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:33:02,079 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:33:02,079 - INFO - Getting canonical form for: 4B
-2025-03-18 21:33:02,079 - INFO - Canonical ID: 4B
-2025-03-18 21:33:02,079 - INFO - All variants: ['4B']
-2025-03-18 21:33:02,079 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:33:02,079 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:33:02,079 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:33:02,079 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:33:02,080 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:33:02,080 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:33:02,080 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:33:02,390 - INFO - Successfully converted data: 3093 records found
-2025-03-18 21:33:02,392 - INFO - Query returned: 3093 records
-2025-03-18 21:33:02,392 - INFO - Converting to DataFrame...
-2025-03-18 21:33:02,396 - INFO - DataFrame created with shape: (3093, 12)
-2025-03-18 21:33:02,396 - INFO - Processing DataFrame...
-2025-03-18 21:33:02,396 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:33:02,396 - INFO - Cleaning latency data...
-2025-03-18 21:33:02,399 - INFO - DataFrame shape after cleaning: (3093, 12)
-2025-03-18 21:33:02,402 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:33:02,403 - INFO - Converting timestamps...
-2025-03-18 21:33:02,418 - INFO - Converting to records...
-2025-03-18 21:33:02,447 - INFO - Created 3093 result records
-2025-03-18 21:33:02,448 - INFO - Returning response with status code 200 and 3093 records
-2025-03-18 21:33:41,730 - INFO - ================================================================================
-2025-03-18 21:33:41,730 - INFO - Data.py script starting
-2025-03-18 21:33:41,731 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:33:41,731 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:33:41,731 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:33:41,731 - INFO - User running script: Unknown
-2025-03-18 21:33:41,731 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:33:41,731 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:33:42,154 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:33:42,154 - INFO - Calling data_endpoint function...
-2025-03-18 21:33:42,154 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:33:42,154 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:33:42,154 - INFO - Getting canonical form for: 4B
-2025-03-18 21:33:42,154 - INFO - Canonical ID: 4B
-2025-03-18 21:33:42,154 - INFO - All variants: ['4B']
-2025-03-18 21:33:42,154 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:33:42,154 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:33:42,154 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:33:42,154 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:33:42,154 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:33:42,155 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:33:42,155 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:33:42,470 - INFO - Successfully converted data: 3094 records found
-2025-03-18 21:33:42,471 - INFO - Query returned: 3094 records
-2025-03-18 21:33:42,471 - INFO - Converting to DataFrame...
-2025-03-18 21:33:42,476 - INFO - DataFrame created with shape: (3094, 12)
-2025-03-18 21:33:42,476 - INFO - Processing DataFrame...
-2025-03-18 21:33:42,476 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:33:42,476 - INFO - Cleaning latency data...
-2025-03-18 21:33:42,478 - INFO - DataFrame shape after cleaning: (3094, 12)
-2025-03-18 21:33:42,481 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:33:42,483 - INFO - Converting timestamps...
-2025-03-18 21:33:42,497 - INFO - Converting to records...
-2025-03-18 21:33:42,525 - INFO - Created 3094 result records
-2025-03-18 21:33:42,526 - INFO - Returning response with status code 200 and 3094 records
-2025-03-18 21:34:05,578 - INFO - ================================================================================
-2025-03-18 21:34:05,578 - INFO - Data.py script starting
-2025-03-18 21:34:05,578 - INFO - Current working directory: /home/ygao/latency/assets/python
-2025-03-18 21:34:05,578 - INFO - Script path: /var/www/html/web_internal/students/ygao/latency/assets/python/data.py
-2025-03-18 21:34:05,578 - INFO - Python version: 3.9.21 (main, Dec 11 2024, 16:24:11)
-[GCC 11.2.0]
-2025-03-18 21:34:05,578 - INFO - User running script: Unknown
-2025-03-18 21:34:05,578 - INFO - Looking for sat_db_functions in: /var/www/html/web_internal/students/ygao/latency/assets/python and /var/www/html/web_internal/students/ygao/latency/assets
-2025-03-18 21:34:05,578 - INFO - Directory contents: ['satellite_relationships.json', '.htaccess', 'latency_viewer.log', 'data.py', 'metadata.py', 'sat_db_functions.py', 'satellites.py', 'generate_relationship.py']
-2025-03-18 21:34:05,987 - INFO - Successfully imported from sat_db_functions
-2025-03-18 21:34:05,987 - INFO - Calling data_endpoint function...
-2025-03-18 21:34:05,988 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 21:34:05,988 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 21:34:05,988 - INFO - Getting canonical form for: 4B
-2025-03-18 21:34:05,988 - INFO - Canonical ID: 4B
-2025-03-18 21:34:05,988 - INFO - All variants: ['4B']
-2025-03-18 21:34:05,988 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 21:34:05,988 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 21:34:05,988 - INFO - About to call run_sat_latency_query...
-2025-03-18 21:34:05,988 - INFO - Querying satellite latency data from 2025-03-18T00:00:00 to 2025-03-18T23:59:59
-2025-03-18 21:34:05,988 - INFO - Converted timestamps: 2025-03-18 00:00:00+00:00 to 2025-03-18 23:59:59+00:00
-2025-03-18 21:34:05,988 - INFO - Query parameters: database=/data/sat_latency, start_date=2025-03-18 00:00:00+00:00, end_date=2025-03-18 23:59:59+00:00
-2025-03-18 21:34:05,988 - INFO - Filters: satellite_ids=['4B'], coverage=None, instrument=['GIIRS']
-2025-03-18 21:34:06,296 - INFO - Successfully converted data: 3094 records found
-2025-03-18 21:34:06,298 - INFO - Query returned: 3094 records
-2025-03-18 21:34:06,298 - INFO - Converting to DataFrame...
-2025-03-18 21:34:06,302 - INFO - DataFrame created with shape: (3094, 12)
-2025-03-18 21:34:06,303 - INFO - Processing DataFrame...
-2025-03-18 21:34:06,303 - INFO - Columns after normalization: ['topic', 'band', 'coverage', 'ingest_source', 'instrument', 'satellite_id', 'section', 'reception_time', 'start_time', 'end_time', 'create_time', 'latency']
-2025-03-18 21:34:06,303 - INFO - Cleaning latency data...
-2025-03-18 21:34:06,305 - INFO - DataFrame shape after cleaning: (3094, 12)
-2025-03-18 21:34:06,308 - INFO - Adding canonical_satellite_id column...
-2025-03-18 21:34:06,310 - INFO - Converting timestamps...
-2025-03-18 21:34:06,325 - INFO - Converting to records...
-2025-03-18 21:34:06,353 - INFO - Created 3094 result records
-2025-03-18 21:34:06,354 - INFO - Returning response with status code 200 and 3094 records
diff --git a/assets/python/metadata.py b/assets/python/metadata.py
index 5d0ae548f68bf160536ced2f4b44f958809748e0..2fff1e3cec4aebb39f8d8a25bf6b9dda18c543b5 100755
--- a/assets/python/metadata.py
+++ b/assets/python/metadata.py
@@ -1,4 +1,4 @@
-#!/home/ygao/miniconda/envs/py39env/bin/python
+#!/home/oper/py39env/bin/python
import cgi
import json
import os
diff --git a/latency_viewer.log b/latency_viewer.log
deleted file mode 100644
index c652b69a6f8b2320437ff21d45ce18c9b89f92b0..0000000000000000000000000000000000000000
--- a/latency_viewer.log
+++ /dev/null
@@ -1,734 +0,0 @@
-2025-03-07 18:54:02,083 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:54:02,084 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:54:02,084 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:54:02,084 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:54:05,587 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-07T00:03:28+00:00", "latency": 776.71}, {"satellite_ID": "...
-2025-03-07 18:54:05,588 - INFO - Successfully parsed JSON data: 77 records found
-2025-03-07 18:54:05,607 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:54:05,608 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:54:05,608 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:54:05,608 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:54:08,696 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-07T00:03:28+00:00", "latency": 776.71}, {"satellite_ID": "...
-2025-03-07 18:54:08,697 - INFO - Successfully parsed JSON data: 77 records found
-2025-03-07 18:54:49,449 - INFO - Expanded satellite ID DMSP-17 to variants: ['DMSP-17', 'dmsp17']
-2025-03-07 18:54:49,449 - INFO - Data request - Period: 2025-03-06T00:00:00 to 2025-03-06T23:59:59, Filters: {'satellite-id': ['DMSP-17', 'dmsp17'], 'instrument': 'mirs'}
-2025-03-07 18:54:49,449 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-06T00:00:00' --until '2025-03-06T23:59:59' --output-type json --satellite-id "DMSP-17" "dmsp17" --instrument "mirs"
-2025-03-07 18:54:49,449 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:54:52,812 - INFO - Command output (first 200 chars): [{"satellite_ID": "dmsp17", "band": null, "coverage": null, "ingest_source": null, "instrument": "mirs", "section": null, "start_time": "2025-03-06T00:50:00+00:00", "latency": 10954.236}, {"satellite_...
-2025-03-07 18:54:52,813 - INFO - Successfully parsed JSON data: 16 records found
-2025-03-07 18:54:52,830 - INFO - Expanded satellite ID DMSP-17 to variants: ['DMSP-17', 'dmsp17']
-2025-03-07 18:54:52,830 - INFO - Data request - Period: 2025-03-06T00:00:00 to 2025-03-06T23:59:59, Filters: {'satellite-id': ['DMSP-17', 'dmsp17'], 'instrument': 'mirs'}
-2025-03-07 18:54:52,830 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-06T00:00:00' --until '2025-03-06T23:59:59' --output-type json --satellite-id "DMSP-17" "dmsp17" --instrument "mirs"
-2025-03-07 18:54:52,830 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:54:56,113 - INFO - Command output (first 200 chars): [{"satellite_ID": "dmsp17", "band": null, "coverage": null, "ingest_source": null, "instrument": "mirs", "section": null, "start_time": "2025-03-06T00:50:00+00:00", "latency": 10954.236}, {"satellite_...
-2025-03-07 18:54:56,113 - INFO - Successfully parsed JSON data: 16 records found
-2025-03-07 18:57:43,163 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:57:43,163 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:57:43,163 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:57:43,163 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:57:46,691 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-07T00:03:28+00:00", "latency": 776.71}, {"satellite_ID": "...
-2025-03-07 18:57:46,691 - INFO - Successfully parsed JSON data: 77 records found
-2025-03-07 18:57:46,713 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:57:46,713 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:57:46,713 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:57:46,713 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-07 18:57:50,278 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-07T00:03:28+00:00", "latency": 776.71}, {"satellite_ID": "...
-2025-03-07 18:57:50,278 - INFO - Successfully parsed JSON data: 77 records found
-2025-03-07 18:58:45,311 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:58:45,311 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:58:45,311 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:58:45,311 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-07 18:58:45,773 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-07 18:58:45,774 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-07 18:58:45,774 - INFO - Data request - Period: 2025-03-07T00:00:00 to 2025-03-07T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-07 18:58:45,774 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-07T00:00:00' --until '2025-03-07T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-07 18:58:45,774 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-07 18:58:46,009 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:13,951 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 15:05:13,951 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 15:05:13,951 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-14T00:00:00' --until '2025-03-14T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 15:05:13,955 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:14,575 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:14,575 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 15:05:14,575 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 15:05:14,576 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-14T00:00:00' --until '2025-03-14T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 15:05:14,576 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:14,874 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:19,639 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 15:05:19,639 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 15:05:19,639 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-13T00:00:00' --until '2025-03-13T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 15:05:19,639 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:19,884 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:19,884 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 15:05:19,884 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 15:05:19,884 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-13T00:00:00' --until '2025-03-13T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 15:05:19,885 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:20,123 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:24,794 - INFO - Expanded satellite ID DMSP-18 to variants: ['DMSP-18', 'dmsp18']
-2025-03-14 15:05:24,795 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['DMSP-18', 'dmsp18'], 'instrument': 'mirs'}
-2025-03-14 15:05:24,795 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-13T00:00:00' --until '2025-03-13T23:59:59' --output-type json --satellite-id "DMSP-18" "dmsp18" --instrument "mirs"
-2025-03-14 15:05:24,795 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:25,033 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:25,034 - INFO - Expanded satellite ID DMSP-18 to variants: ['DMSP-18', 'dmsp18']
-2025-03-14 15:05:25,034 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['DMSP-18', 'dmsp18'], 'instrument': 'mirs'}
-2025-03-14 15:05:25,034 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-13T00:00:00' --until '2025-03-13T23:59:59' --output-type json --satellite-id "DMSP-18" "dmsp18" --instrument "mirs"
-2025-03-14 15:05:25,034 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:25,281 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:30,986 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:30,986 - INFO - Data request - Period: 2025-03-09T00:00:00 to 2025-03-09T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:05:30,986 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-09T00:00:00' --until '2025-03-09T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:05:30,987 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:31,276 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:31,277 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:31,277 - INFO - Data request - Period: 2025-03-09T00:00:00 to 2025-03-09T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:05:31,277 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-09T00:00:00' --until '2025-03-09T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:05:31,277 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:31,516 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:49,031 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:49,032 - INFO - Data request - Period: 2025-03-09T00:00:00 to 2025-03-09T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'SUVI'}
-2025-03-14 15:05:49,032 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-09T00:00:00' --until '2025-03-09T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "SUVI"
-2025-03-14 15:05:49,032 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:49,285 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:49,286 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:49,286 - INFO - Data request - Period: 2025-03-09T00:00:00 to 2025-03-09T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'SUVI'}
-2025-03-14 15:05:49,286 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-09T00:00:00' --until '2025-03-09T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "SUVI"
-2025-03-14 15:05:49,286 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:49,540 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:55,280 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:55,281 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:05:55,281 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:05:55,281 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:55,519 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:05:55,519 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:05:55,520 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:05:55,520 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:05:55,520 - INFO - Executing: sudo -u oper -i scripts/run_sat_latency.sh
-2025-03-14 15:05:55,763 - ERROR - Command failed with exit code 127: -bash: scripts/run_sat_latency.sh: No such file or directory
-
-2025-03-14 15:06:31,542 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:06:31,543 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:06:31,543 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:06:31,543 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 15:06:48,616 - INFO - Command output (first 200 chars): [{"satellite_ID": "G16", "band": "4", "coverage": "Mesoscale-1", "ingest_source": "inge ABI GRB-R v1.0.0 : grbdelta.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:0...
-2025-03-14 15:06:49,312 - INFO - Successfully parsed JSON data: 331542 records found
-2025-03-14 15:06:56,167 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:06:56,168 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:06:56,168 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:06:56,168 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 15:07:03,352 - INFO - Command output (first 200 chars): [{"satellite_ID": "G16", "band": "4", "coverage": "Mesoscale-1", "ingest_source": "inge ABI GRB-R v1.0.0 : grbdelta.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:0...
-2025-03-14 15:07:03,984 - INFO - Successfully parsed JSON data: 331542 records found
-2025-03-14 15:35:46,521 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:35:46,522 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:35:46,522 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:35:46,522 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 15:35:54,493 - INFO - Command output (first 200 chars): [{"satellite_ID": "G16", "band": "4", "coverage": "Mesoscale-1", "ingest_source": "inge ABI GRB-R v1.0.0 : grbdelta.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:0...
-2025-03-14 15:35:55,132 - INFO - Successfully parsed JSON data: 331542 records found
-2025-03-14 15:36:01,200 - INFO - Expanded satellite ID G16 to variants: ['G16', 'g16']
-2025-03-14 15:36:01,201 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G16', 'g16'], 'instrument': 'ABI'}
-2025-03-14 15:36:01,201 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G16" "g16" --instrument "ABI"
-2025-03-14 15:36:01,201 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 15:36:07,406 - INFO - Command output (first 200 chars): [{"satellite_ID": "G16", "band": "4", "coverage": "Mesoscale-1", "ingest_source": "inge ABI GRB-R v1.0.0 : grbdelta.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:0...
-2025-03-14 15:36:08,024 - INFO - Successfully parsed JSON data: 331542 records found
-2025-03-14 16:25:53,769 - INFO - Expanded satellite ID G18 to variants: ['G18', 'g18']
-2025-03-14 16:25:53,770 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G18', 'g18'], 'coverage': 'CONUS', 'instrument': 'ABI'}
-2025-03-14 16:25:53,770 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G18" "g18" --coverage "CONUS" --instrument "ABI"
-2025-03-14 16:25:53,770 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 16:25:59,305 - INFO - Command output (first 200 chars): [{"satellite_ID": "G18", "band": "2", "coverage": "CONUS", "ingest_source": "inge ABI GRB-R v1.0.0 : grbgamma.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:01:17.7...
-2025-03-14 16:25:59,362 - INFO - Successfully parsed JSON data: 28373 records found
-2025-03-14 16:25:59,858 - INFO - Expanded satellite ID G18 to variants: ['G18', 'g18']
-2025-03-14 16:25:59,858 - INFO - Data request - Period: 2025-03-08T00:00:00 to 2025-03-08T23:59:59, Filters: {'satellite-id': ['G18', 'g18'], 'coverage': 'CONUS', 'instrument': 'ABI'}
-2025-03-14 16:25:59,858 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-08T00:00:00' --until '2025-03-08T23:59:59' --output-type json --satellite-id "G18" "g18" --coverage "CONUS" --instrument "ABI"
-2025-03-14 16:25:59,858 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 16:26:03,418 - INFO - Command output (first 200 chars): [{"satellite_ID": "G18", "band": "2", "coverage": "CONUS", "ingest_source": "inge ABI GRB-R v1.0.0 : grbgamma.ssec.wisc.edu", "instrument": "ABI", "section": null, "start_time": "2025-03-08T00:01:17.7...
-2025-03-14 16:26:03,487 - INFO - Successfully parsed JSON data: 28373 records found
-2025-03-14 17:27:16,411 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 17:27:16,411 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 17:27:16,411 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-14T00:00:00' --until '2025-03-14T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 17:27:16,412 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 17:27:22,241 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-14T00:03:28+00:00", "latency": 1623.258}, {"satellite_ID":...
-2025-03-14 17:27:22,246 - INFO - Successfully parsed JSON data: 2583 records found
-2025-03-14 17:27:22,311 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 17:27:22,311 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 17:27:22,311 - INFO - Running command: module load miniconda/3.6-base && source activate ~/.mdrexler_conda && sat_latency_interface -d /data/sat_latency --from '2025-03-14T00:00:00' --until '2025-03-14T23:59:59' --output-type json --satellite-id "4B" --instrument "GIIRS"
-2025-03-14 17:27:22,311 - INFO - Executing: sudo -u oper -i /tmp/run_sat_latency.sh
-2025-03-14 17:27:25,803 - INFO - Command output (first 200 chars): [{"satellite_ID": "4B", "band": null, "coverage": null, "ingest_source": null, "instrument": "GIIRS", "section": null, "start_time": "2025-03-14T00:03:28+00:00", "latency": 1623.258}, {"satellite_ID":...
-2025-03-14 17:27:25,808 - INFO - Successfully parsed JSON data: 2587 records found
-2025-03-14 17:53:03,576 - INFO - ================================================================================
-2025-03-14 17:53:03,576 - INFO - Data.py script starting
-2025-03-14 17:53:03,576 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:53:03,576 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:53:03,576 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:53:03,576 - INFO - User running script: ygao
-2025-03-14 17:53:03,576 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin
-2025-03-14 17:53:03,576 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:53:03,578 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:53:03,578 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 46, in <module>
- from sat_db_functions import run_sat_latency_query, get_canonical_id, get_all_variants
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:53:03,578 - INFO - Calling data_endpoint function...
-2025-03-14 17:53:03,578 - INFO - Query parameters: start_date=2025-03-14, end_date=2025-03-14, start_hour=00:00, end_hour=23:59
-2025-03-14 17:53:03,578 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-14 17:53:03,579 - INFO - Getting canonical form for: 4B
-2025-03-14 17:53:03,579 - ERROR - Error processing data request: name 'get_canonical_id' is not defined
-2025-03-14 17:53:03,579 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 85, in data_endpoint
- canonical_id = get_canonical_id(satellite_id)
-NameError: name 'get_canonical_id' is not defined
-
-2025-03-14 17:53:03,579 - WARNING - Returning error with status code 500: {'message': "Internal Server Error: name 'get_canonical_id' is not defined", 'data': []}
-2025-03-14 17:53:03,579 - INFO - Returning response with status code 500 and 0 records
-2025-03-14 17:53:04,799 - INFO - ================================================================================
-2025-03-14 17:53:04,799 - INFO - Data.py script starting
-2025-03-14 17:53:04,799 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:53:04,799 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:53:04,799 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:53:04,799 - INFO - User running script: ygao
-2025-03-14 17:53:04,799 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin
-2025-03-14 17:53:04,799 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:53:04,801 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:53:04,801 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 46, in <module>
- from sat_db_functions import run_sat_latency_query, get_canonical_id, get_all_variants
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:53:04,801 - INFO - Calling data_endpoint function...
-2025-03-14 17:53:04,801 - INFO - Query parameters: start_date=2025-03-14, end_date=2025-03-14, start_hour=00:00, end_hour=23:59
-2025-03-14 17:53:04,801 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-14 17:53:04,802 - INFO - Getting canonical form for: 4B
-2025-03-14 17:53:04,802 - ERROR - Error processing data request: name 'get_canonical_id' is not defined
-2025-03-14 17:53:04,802 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 85, in data_endpoint
- canonical_id = get_canonical_id(satellite_id)
-NameError: name 'get_canonical_id' is not defined
-
-2025-03-14 17:53:04,802 - WARNING - Returning error with status code 500: {'message': "Internal Server Error: name 'get_canonical_id' is not defined", 'data': []}
-2025-03-14 17:53:04,802 - INFO - Returning response with status code 500 and 0 records
-2025-03-14 17:53:07,742 - INFO - ================================================================================
-2025-03-14 17:53:07,742 - INFO - Data.py script starting
-2025-03-14 17:53:07,742 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:53:07,742 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:53:07,742 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:53:07,742 - INFO - User running script: ygao
-2025-03-14 17:53:07,742 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin
-2025-03-14 17:53:07,742 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:53:07,743 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:53:07,744 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 46, in <module>
- from sat_db_functions import run_sat_latency_query, get_canonical_id, get_all_variants
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:53:07,744 - INFO - Calling data_endpoint function...
-2025-03-14 17:53:07,744 - INFO - Query parameters: start_date=2025-03-13, end_date=2025-03-13, start_hour=00:00, end_hour=23:59
-2025-03-14 17:53:07,744 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-14 17:53:07,744 - INFO - Getting canonical form for: G16
-2025-03-14 17:53:07,744 - ERROR - Error processing data request: name 'get_canonical_id' is not defined
-2025-03-14 17:53:07,744 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 85, in data_endpoint
- canonical_id = get_canonical_id(satellite_id)
-NameError: name 'get_canonical_id' is not defined
-
-2025-03-14 17:53:07,744 - WARNING - Returning error with status code 500: {'message': "Internal Server Error: name 'get_canonical_id' is not defined", 'data': []}
-2025-03-14 17:53:07,744 - INFO - Returning response with status code 500 and 0 records
-2025-03-14 17:54:31,646 - INFO - ================================================================================
-2025-03-14 17:54:31,646 - INFO - Data.py script starting
-2025-03-14 17:54:31,646 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:54:31,646 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:54:31,646 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:54:31,646 - INFO - User running script: ygao
-2025-03-14 17:54:31,646 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin
-2025-03-14 17:54:31,646 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:54:31,649 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:54:31,649 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 46, in <module>
- from sat_db_functions import run_sat_latency_query, get_canonical_id, get_all_variants
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:54:31,649 - INFO - Calling data_endpoint function...
-2025-03-14 17:54:31,650 - INFO - Query parameters: start_date=2025-03-14, end_date=2025-03-14, start_hour=00:00, end_hour=23:59
-2025-03-14 17:54:31,650 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-14 17:54:31,650 - INFO - Getting canonical form for: 4B
-2025-03-14 17:54:31,650 - ERROR - Error processing data request: name 'get_canonical_id' is not defined
-2025-03-14 17:54:31,650 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 85, in data_endpoint
- canonical_id = get_canonical_id(satellite_id)
-NameError: name 'get_canonical_id' is not defined
-
-2025-03-14 17:54:31,650 - WARNING - Returning error with status code 500: {'message': "Internal Server Error: name 'get_canonical_id' is not defined", 'data': []}
-2025-03-14 17:54:31,650 - INFO - Returning response with status code 500 and 0 records
-2025-03-14 17:56:35,198 - INFO - ================================================================================
-2025-03-14 17:56:35,198 - INFO - Data.py script starting
-2025-03-14 17:56:35,198 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:56:35,199 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:56:35,199 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:56:35,199 - INFO - User running script: ygao
-2025-03-14 17:56:35,199 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:56:35,199 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:56:35,200 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:56:35,201 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:56:35,201 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:56:35,201 - INFO - Calling data_endpoint function...
-2025-03-14 17:56:35,202 - INFO - Query parameters: start_date=2025-03-14, end_date=2025-03-14, start_hour=00:00, end_hour=23:59
-2025-03-14 17:56:35,202 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-14 17:56:35,202 - INFO - Getting canonical form for: 4B
-2025-03-14 17:56:35,202 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-14 17:56:35,202 - INFO - Canonical ID: 4B
-2025-03-14 17:56:35,202 - WARNING - Using fallback get_all_variants for 4B
-2025-03-14 17:56:35,202 - INFO - All variants: ['4B']
-2025-03-14 17:56:35,202 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 17:56:35,202 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 17:56:35,202 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:56:35,202 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:56:35,202 - INFO - Query returned: 0 records
-2025-03-14 17:56:35,202 - INFO - Query returned no data
-2025-03-14 17:56:35,202 - INFO - Returning response with status code 200 and 0 records
-2025-03-14 17:56:36,810 - INFO - ================================================================================
-2025-03-14 17:56:36,810 - INFO - Data.py script starting
-2025-03-14 17:56:36,810 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:56:36,810 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:56:36,810 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:56:36,810 - INFO - User running script: ygao
-2025-03-14 17:56:36,810 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:56:36,810 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:56:36,811 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:56:36,812 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:56:36,812 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:56:36,812 - INFO - Calling data_endpoint function...
-2025-03-14 17:56:36,812 - INFO - Query parameters: start_date=2025-03-14, end_date=2025-03-14, start_hour=00:00, end_hour=23:59
-2025-03-14 17:56:36,812 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-14 17:56:36,812 - INFO - Getting canonical form for: 4B
-2025-03-14 17:56:36,812 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-14 17:56:36,812 - INFO - Canonical ID: 4B
-2025-03-14 17:56:36,812 - WARNING - Using fallback get_all_variants for 4B
-2025-03-14 17:56:36,812 - INFO - All variants: ['4B']
-2025-03-14 17:56:36,812 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-14 17:56:36,812 - INFO - Data request - Period: 2025-03-14T00:00:00 to 2025-03-14T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-14 17:56:36,812 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:56:36,812 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:56:36,812 - INFO - Query returned: 0 records
-2025-03-14 17:56:36,812 - INFO - Query returned no data
-2025-03-14 17:56:36,812 - INFO - Returning response with status code 200 and 0 records
-2025-03-14 17:56:58,485 - INFO - ================================================================================
-2025-03-14 17:56:58,485 - INFO - Data.py script starting
-2025-03-14 17:56:58,485 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:56:58,485 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:56:58,485 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:56:58,485 - INFO - User running script: ygao
-2025-03-14 17:56:58,485 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:56:58,485 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:56:58,487 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:56:58,487 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:56:58,487 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:56:58,487 - INFO - Calling data_endpoint function...
-2025-03-14 17:56:58,487 - INFO - Query parameters: start_date=2025-03-13, end_date=2025-03-13, start_hour=00:00, end_hour=23:59
-2025-03-14 17:56:58,488 - INFO - Filter parameters: satellite_id=DMSP-16, coverage=None, instrument=ssmi
-2025-03-14 17:56:58,488 - INFO - Getting canonical form for: DMSP-16
-2025-03-14 17:56:58,488 - WARNING - Using fallback get_canonical_id for DMSP-16
-2025-03-14 17:56:58,488 - INFO - Canonical ID: DMSP-16
-2025-03-14 17:56:58,488 - WARNING - Using fallback get_all_variants for DMSP-16
-2025-03-14 17:56:58,488 - INFO - All variants: ['DMSP-16']
-2025-03-14 17:56:58,488 - INFO - Expanded satellite ID DMSP-16 to variants: ['DMSP-16']
-2025-03-14 17:56:58,488 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['DMSP-16'], 'instrument': 'ssmi'}
-2025-03-14 17:56:58,488 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:56:58,488 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:56:58,488 - INFO - Query returned: 0 records
-2025-03-14 17:56:58,488 - INFO - Query returned no data
-2025-03-14 17:56:58,488 - INFO - Returning response with status code 200 and 0 records
-2025-03-14 17:57:02,043 - INFO - ================================================================================
-2025-03-14 17:57:02,043 - INFO - Data.py script starting
-2025-03-14 17:57:02,043 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:57:02,043 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:57:02,043 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:57:02,043 - INFO - User running script: ygao
-2025-03-14 17:57:02,043 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:57:02,043 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:57:02,044 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:57:02,045 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:57:02,045 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:57:02,045 - INFO - Calling data_endpoint function...
-2025-03-14 17:57:02,045 - INFO - Query parameters: start_date=2025-03-13, end_date=2025-03-13, start_hour=00:00, end_hour=23:59
-2025-03-14 17:57:02,045 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-14 17:57:02,045 - INFO - Getting canonical form for: G16
-2025-03-14 17:57:02,045 - WARNING - Using fallback get_canonical_id for G16
-2025-03-14 17:57:02,045 - INFO - Canonical ID: G16
-2025-03-14 17:57:02,045 - WARNING - Using fallback get_all_variants for G16
-2025-03-14 17:57:02,045 - INFO - All variants: ['G16']
-2025-03-14 17:57:02,045 - INFO - Expanded satellite ID G16 to variants: ['G16']
-2025-03-14 17:57:02,045 - INFO - Data request - Period: 2025-03-13T00:00:00 to 2025-03-13T23:59:59, Filters: {'satellite-id': ['G16'], 'instrument': 'ABI'}
-2025-03-14 17:57:02,045 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:57:02,045 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:57:02,045 - INFO - Query returned: 0 records
-2025-03-14 17:57:02,045 - INFO - Query returned no data
-2025-03-14 17:57:02,045 - INFO - Returning response with status code 200 and 0 records
-2025-03-14 17:57:05,682 - INFO - ================================================================================
-2025-03-14 17:57:05,682 - INFO - Data.py script starting
-2025-03-14 17:57:05,682 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:57:05,682 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:57:05,682 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:57:05,682 - INFO - User running script: ygao
-2025-03-14 17:57:05,682 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:57:05,682 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:57:05,684 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:57:05,684 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:57:05,684 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:57:05,684 - INFO - Calling data_endpoint function...
-2025-03-14 17:57:05,684 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-14 17:57:05,685 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=ABI
-2025-03-14 17:57:05,685 - INFO - Getting canonical form for: G16
-2025-03-14 17:57:05,685 - WARNING - Using fallback get_canonical_id for G16
-2025-03-14 17:57:05,685 - INFO - Canonical ID: G16
-2025-03-14 17:57:05,685 - WARNING - Using fallback get_all_variants for G16
-2025-03-14 17:57:05,685 - INFO - All variants: ['G16']
-2025-03-14 17:57:05,685 - INFO - Expanded satellite ID G16 to variants: ['G16']
-2025-03-14 17:57:05,685 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['G16'], 'instrument': 'ABI'}
-2025-03-14 17:57:05,685 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:57:05,685 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:57:05,685 - INFO - Query returned: 0 records
-2025-03-14 17:57:05,685 - INFO - Query returned no data
-2025-03-14 17:57:05,685 - INFO - Returning response with status code 200 and 0 records
-2025-03-14 17:57:07,557 - INFO - ================================================================================
-2025-03-14 17:57:07,557 - INFO - Data.py script starting
-2025-03-14 17:57:07,557 - INFO - Current working directory: /home/ygao/latency
-2025-03-14 17:57:07,557 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-14 17:57:07,557 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-14 17:57:07,557 - INFO - User running script: ygao
-2025-03-14 17:57:07,557 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-14 17:57:07,557 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-14 17:57:07,559 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-14 17:57:07,559 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-14 17:57:07,559 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-14 17:57:07,559 - INFO - Calling data_endpoint function...
-2025-03-14 17:57:07,559 - INFO - Query parameters: start_date=2025-03-12, end_date=2025-03-12, start_hour=00:00, end_hour=23:59
-2025-03-14 17:57:07,560 - INFO - Filter parameters: satellite_id=G16, coverage=None, instrument=MAG
-2025-03-14 17:57:07,560 - INFO - Getting canonical form for: G16
-2025-03-14 17:57:07,560 - WARNING - Using fallback get_canonical_id for G16
-2025-03-14 17:57:07,560 - INFO - Canonical ID: G16
-2025-03-14 17:57:07,560 - WARNING - Using fallback get_all_variants for G16
-2025-03-14 17:57:07,560 - INFO - All variants: ['G16']
-2025-03-14 17:57:07,560 - INFO - Expanded satellite ID G16 to variants: ['G16']
-2025-03-14 17:57:07,560 - INFO - Data request - Period: 2025-03-12T00:00:00 to 2025-03-12T23:59:59, Filters: {'satellite-id': ['G16'], 'instrument': 'MAG'}
-2025-03-14 17:57:07,560 - INFO - About to call run_sat_latency_query...
-2025-03-14 17:57:07,560 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-14 17:57:07,560 - INFO - Query returned: 0 records
-2025-03-14 17:57:07,560 - INFO - Query returned no data
-2025-03-14 17:57:07,560 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:26:08,690 - INFO - ================================================================================
-2025-03-18 18:26:08,691 - INFO - Data.py script starting
-2025-03-18 18:26:08,691 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:26:08,691 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:26:08,691 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:26:08,691 - INFO - User running script: ygao
-2025-03-18 18:26:08,691 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:26:08,691 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:26:08,693 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-18 18:26:08,694 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-18 18:26:08,694 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:26:08,694 - INFO - Calling data_endpoint function...
-2025-03-18 18:26:08,694 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:26:08,694 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:26:08,694 - INFO - Getting canonical form for: 4B
-2025-03-18 18:26:08,694 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:26:08,694 - INFO - Canonical ID: 4B
-2025-03-18 18:26:08,694 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:26:08,694 - INFO - All variants: ['4B']
-2025-03-18 18:26:08,694 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:26:08,694 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:26:08,694 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:26:08,694 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:26:08,695 - INFO - Query returned: 0 records
-2025-03-18 18:26:08,695 - INFO - Query returned no data
-2025-03-18 18:26:08,695 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:26:10,659 - INFO - ================================================================================
-2025-03-18 18:26:10,659 - INFO - Data.py script starting
-2025-03-18 18:26:10,659 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:26:10,659 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:26:10,659 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:26:10,659 - INFO - User running script: ygao
-2025-03-18 18:26:10,660 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:26:10,660 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:26:10,661 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-18 18:26:10,661 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-18 18:26:10,661 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:26:10,661 - INFO - Calling data_endpoint function...
-2025-03-18 18:26:10,662 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:26:10,662 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:26:10,662 - INFO - Getting canonical form for: 4B
-2025-03-18 18:26:10,662 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:26:10,662 - INFO - Canonical ID: 4B
-2025-03-18 18:26:10,662 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:26:10,662 - INFO - All variants: ['4B']
-2025-03-18 18:26:10,662 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:26:10,662 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:26:10,662 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:26:10,662 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:26:10,662 - INFO - Query returned: 0 records
-2025-03-18 18:26:10,662 - INFO - Query returned no data
-2025-03-18 18:26:10,662 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:29:21,202 - INFO - ================================================================================
-2025-03-18 18:29:21,202 - INFO - Data.py script starting
-2025-03-18 18:29:21,202 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:29:21,202 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:29:21,202 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:29:21,202 - INFO - User running script: ygao
-2025-03-18 18:29:21,202 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:29:21,202 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:29:21,206 - ERROR - Unexpected error during import: future feature annotations is not defined (database.py, line 1)
-2025-03-18 18:29:21,206 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 13, in <module>
- from sat_latency.database import (
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/database.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-18 18:29:21,206 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:29:21,206 - INFO - Calling data_endpoint function...
-2025-03-18 18:29:21,206 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:29:21,206 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:29:21,206 - INFO - Getting canonical form for: 4B
-2025-03-18 18:29:21,207 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:29:21,207 - INFO - Canonical ID: 4B
-2025-03-18 18:29:21,207 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:29:21,207 - INFO - All variants: ['4B']
-2025-03-18 18:29:21,207 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:29:21,207 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:29:21,207 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:29:21,207 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:29:21,207 - INFO - Query returned: 0 records
-2025-03-18 18:29:21,207 - INFO - Query returned no data
-2025-03-18 18:29:21,207 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:30:26,787 - INFO - ================================================================================
-2025-03-18 18:30:26,787 - INFO - Data.py script starting
-2025-03-18 18:30:26,787 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:30:26,787 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:30:26,787 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:30:26,787 - INFO - User running script: ygao
-2025-03-18 18:30:26,787 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:30:26,788 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:30:26,792 - ERROR - Import error: No module named 'attrs'
-2025-03-18 18:30:26,793 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 13, in <module>
- from sat_latency.database import (
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/database.py", line 9, in <module>
- from attrs import define, field
-ModuleNotFoundError: No module named 'attrs'
-
-2025-03-18 18:30:26,793 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:30:26,793 - INFO - Calling data_endpoint function...
-2025-03-18 18:30:26,793 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:30:26,793 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:30:26,793 - INFO - Getting canonical form for: 4B
-2025-03-18 18:30:26,793 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:30:26,793 - INFO - Canonical ID: 4B
-2025-03-18 18:30:26,793 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:30:26,793 - INFO - All variants: ['4B']
-2025-03-18 18:30:26,793 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:30:26,793 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:30:26,793 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:30:26,793 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:30:26,793 - INFO - Query returned: 0 records
-2025-03-18 18:30:26,794 - INFO - Query returned no data
-2025-03-18 18:30:26,794 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:31:24,306 - INFO - ================================================================================
-2025-03-18 18:31:24,306 - INFO - Data.py script starting
-2025-03-18 18:31:24,306 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:31:24,306 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:31:24,306 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:31:24,306 - INFO - User running script: ygao
-2025-03-18 18:31:24,306 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:31:24,306 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:31:24,327 - ERROR - Unexpected error during import: unsupported operand type(s) for |: 'type' and 'NoneType'
-2025-03-18 18:31:24,328 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 13, in <module>
- from sat_latency.database import (
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/database.py", line 12, in <module>
- from sat_latency._utils import daterange
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/_utils.py", line 35, in <module>
- day_step: int | None = None,
-TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'
-
-2025-03-18 18:31:24,328 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:31:24,328 - INFO - Calling data_endpoint function...
-2025-03-18 18:31:24,328 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:31:24,328 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:31:24,328 - INFO - Getting canonical form for: 4B
-2025-03-18 18:31:24,328 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:31:24,328 - INFO - Canonical ID: 4B
-2025-03-18 18:31:24,328 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:31:24,328 - INFO - All variants: ['4B']
-2025-03-18 18:31:24,328 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:31:24,328 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:31:24,328 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:31:24,328 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:31:24,328 - INFO - Query returned: 0 records
-2025-03-18 18:31:24,328 - INFO - Query returned no data
-2025-03-18 18:31:24,329 - INFO - Returning response with status code 200 and 0 records
-2025-03-18 18:32:32,660 - INFO - ================================================================================
-2025-03-18 18:32:32,660 - INFO - Data.py script starting
-2025-03-18 18:32:32,661 - INFO - Current working directory: /home/ygao/latency
-2025-03-18 18:32:32,661 - INFO - Script path: /home/ygao/latency/cgi-bin/data.py
-2025-03-18 18:32:32,661 - INFO - Python version: 3.6.8 (default, Dec 4 2024, 12:35:02)
-[GCC 8.5.0 20210514 (Red Hat 8.5.0-22)]
-2025-03-18 18:32:32,661 - INFO - User running script: ygao
-2025-03-18 18:32:32,661 - INFO - Looking for sat_db_functions in: /home/ygao/latency/cgi-bin and /home/ygao/latency
-2025-03-18 18:32:32,661 - INFO - Directory contents: ['satellites.py', 'metadata.py', 'data.py', 'sat_db_functions.py', 'satellite_relationships.json', '__pycache__', '.htaccess']
-2025-03-18 18:32:32,662 - ERROR - Unexpected error during import: future feature annotations is not defined (interface.py, line 1)
-2025-03-18 18:32:32,662 - ERROR - Traceback (most recent call last):
- File "/home/ygao/latency/cgi-bin/data.py", line 74, in <module>
- import sat_db_functions
- File "/home/ygao/latency/cgi-bin/sat_db_functions.py", line 6, in <module>
- from sat_latency.interface import satellite_data_from_filters
- File "/home/ygao/latency/venv/lib64/python3.6/site-packages/sat_latency/interface.py", line 1
- from __future__ import annotations
- ^
-SyntaxError: future feature annotations is not defined
-
-2025-03-18 18:32:32,663 - ERROR - Will use fallback functions that provide limited functionality
-2025-03-18 18:32:32,663 - INFO - Calling data_endpoint function...
-2025-03-18 18:32:32,663 - INFO - Query parameters: start_date=2025-03-18, end_date=2025-03-18, start_hour=00:00, end_hour=23:59
-2025-03-18 18:32:32,663 - INFO - Filter parameters: satellite_id=4B, coverage=None, instrument=GIIRS
-2025-03-18 18:32:32,663 - INFO - Getting canonical form for: 4B
-2025-03-18 18:32:32,663 - WARNING - Using fallback get_canonical_id for 4B
-2025-03-18 18:32:32,663 - INFO - Canonical ID: 4B
-2025-03-18 18:32:32,663 - WARNING - Using fallback get_all_variants for 4B
-2025-03-18 18:32:32,663 - INFO - All variants: ['4B']
-2025-03-18 18:32:32,663 - INFO - Expanded satellite ID 4B to variants: ['4B']
-2025-03-18 18:32:32,663 - INFO - Data request - Period: 2025-03-18T00:00:00 to 2025-03-18T23:59:59, Filters: {'satellite-id': ['4B'], 'instrument': 'GIIRS'}
-2025-03-18 18:32:32,663 - INFO - About to call run_sat_latency_query...
-2025-03-18 18:32:32,663 - ERROR - Using fallback run_sat_latency_query - no data will be returned
-2025-03-18 18:32:32,663 - INFO - Query returned: 0 records
-2025-03-18 18:32:32,663 - INFO - Query returned no data
-2025-03-18 18:32:32,663 - INFO - Returning response with status code 200 and 0 records