Report performance statistics to AWS CloudWatch
Created by: dismantl
The spider and scraper component currently record several metrics that can be used to evaluate performance. These metrics should be sent to CloudWatch so they can be graphed and monitored in a dashboard.
For each spider search item, the following metrics are recorded:
- Number of returned results
- Timestamp when search item was queried from MJCS
- Number of seconds for the query to return results
For each run of the spider, the following metrics are recorded:
- Start and end date
- Search criteria
- Duration of run
- Number of queue items still active
- Number of queue items finished
- Number of new cases added to database
- Number of queries submitted to MJCS
For each scrape, the following metrics are recorded:
- Timestamp of scrape
- Number of seconds to complete scrape