Throughout the years Vote Smartians have written lots of quality control queries. We decided that some of those queries should be run on a regular basis. This is basically what
PyQual is - a list of queries that are set up to automatically run on a daily, weekly, or monthly basis. More specifically, they are Sequel (SQL) queries wrapped in python.
The results of those queries are set up to automatically be emailed in spreadsheet format to designated recipients. For example, on a weekly basis the results of the "SIGs without categories" query can be sent to both the Elections Research Director and a Research Associate in SIGs. The Research Directors designate who receives what results using
this tool. If you have permissions, please do not alter who receives what results unless you are a Research Director.
If
PyQual is working properly and you are designated to receive query results, then you should be periodically receiving emails with spreadsheet attachments. If there are no errors, the spreadsheet will be empty. Those spreadsheets contain lists of errors (query results). It's your job to find those errors in the database and fix them. For example, if you're in Speeches, you may receive the "Future Speeches" query results. This will be a spreadsheet full of speeches that have speechdates later than today, which is obviously not possible. It's your job to correct those errors. In this example, you would do research to find the correct speechdate, then fix the error in Admin. It's usually not a pressing matter to fix these errors, so don't worry about getting to them immediately if you have more important things to take care of. If appropriate, don't hesitate to delegate the task of fixing these errors to interns.
| Test ID | Test Name | Program | Frequency | Results |
|---|
| 10 | Multiple Candidates In Congstatus | Key Votes | Daily | Candidates that voted twice on same bill (multiple congstatus_id's) |
| 14 | Orphan Ratings | SIGs | Weekly | Ratings not associated with a candidate (no rating_candidate_id) |
| 25 | Missing Left Office | Bios | Weekly | Federal and state candidates who have left office (inactive or resigned) but have no term-end date |
| 17 | Missing submitted additional bio data | Issue Positions | Weekly | Candidates that have submitted candidate submitted bios, but whose bios have not been added into admin |
| 26 | No synopsis after 6 months | Key Votes | Weekly | Bills that are more than 6 months old and have no synopsis |
| 22 | Officials Missing Dates | Bios | Weekly | Elected officials without a lastelect, firstelect, or nextelect date |
| 5 | Orphan Candidate Check | Bios | Weekly | Candidates that have no associated office (office_candidate_id) and no associated election (election_candidate_id) |
| 21 | Notable Contacts | Issue Positions | Weekly | Candidates we have contacted 8 or more days ago and have been marked as notable |
| 12 | Orphan SIGs | SIGs | Weekly | SIGs that have no associated rating (rating_id) and no associated endorsement (endorse_id) |
| 24 | Bad Next Elect | Elections Monitoring | Monthly | Candidates whose nextelect date is prior to now |
| 11 | Elections Without Stages | Elections Monitoring | Weekly | Elections without an electionstage |
| 7 | KV Without Statuses | Key Votes | Weekly | Bills without statuses (missing congstatus_id) |
| 27 | SIGs without categories | SIGs | Weekly | SIGs that have not been assigned categories |
| 13 | Orphan Offices | Bios | Weekly | Offices that have no associated candidates (no office_candidate_id and no election_candidate_id) |
| 6 | Unreleased Election Results | Elections Monitoring | Daily | Elections that have occurred but have no live results |
| 9 | Future Statuses | Key Votes | Daily | Bills that have a status in the future |
| 19 | Ratings without categories | SIGs | Weekly | Ratings that have not been assigned categories |
| 8 | Missing Finance IDs | Campaign Finance | Daily | Candidates from this year's election cycle without campaign finance information (no finsource_candidate_id) |
| 23 | Missing Term Start After Election | Bios | Monthly | Candidates who have won a general election more than six months ago and still lack a termstart date |
| 16 | Candidates missing ballotname | Bios | Weekly | Candidates in elections post 2012 that have no ballotname |
| 30 | Unreleased Endorsements | SIGs | Monthly | Endorsements collected between 30 and 60 days ago and are not released live |
| 33 | Unreleased Ratings | SIGs | Monthly | Ratings created within the last thirty days and are not released live |
| 34 | Unreleased SIGs with Content | SIGs | Monthly | Unreleased SIGs with a released rating or endorsement |
| 35 | Future Speeches | Speeches | Weekly | Speeches with a speechdate later than today |
| 32 | Non-Percentage Ratings | SIGs | Weekly | Ratings created within the past week that use SIG rating (not our rating), and the rating format is not numeric |
| 31 | Blank Ratings | SIGs | Weekly | Ratings created more than a week ago that either use ourrating and ourrating is null or use sigrating and sigrating is null |
| 29 | Duplicate Candidate Check | Bios | Monthly | Finds candidates with the same firstname and lastname but have different candidate_ids |
| 15 | Candidate Election stage party assignment | Bios | Weekly | Federal or state officials elected post 2012 that have no party affiliation in their election (no electionstage_candidate_party_id) |
| 18 | Candidate missing stages for election | Bios | Daily | Candidates from 2007 and up that are not listed as in an election stage (no electionstage_candidate_id) |
| 28 | Unreleased Elections | Elections Monitoring | Daily | Elections that occurred a week or more ago and are unreleased |
The queries can be found in the drive. Path: Research > Active > Department Management for Directors > QC > pyqual Queries. They can also be found in the recipient
tool mentioned above. Again, please do not alter anything unless you are a Research Director.
As of October 2016, query results are generally not being sent via email due to some sort of error. Mike Shultz, our IT Director, is aware of this and is okay with being bugged to fix this system and periodically being asked to manually force those emails to be sent. The Research Director and IT Director should keep on top of this project and not let it get lost in the abyss of Vote Smart. The program is very useful and should be continued.
