IT Operations Overview
IT Job Descriptions
IT Responsibilities and Priorities
A description of how we make updates to our data on the web site:
Most of the political data is updated by staff in Admin and comes out on the site. Some of the static content on pages like Government 101 are currently in the html templates. Content in the html templates is updated by IT staff upon request from staff. (GitHub is used for source control as of 4/2021)
Outside of our political data, static content on our website is updated in the html templates by request, while some of the more dynamic content (like the staff roster, adviser roster, and blog posts) are updated by staff directly through Django Admin.
A list of tools, platforms, software, or systems related to IT responsibilities for which we have a license:
- Windows and other Microsoft licenses
- Donorperfect
- Aristotle DMDS
- USPS census data
- Google Apps for business, educational orgs
- Google Analytics
Our website requires the most support. As the “face” of the organization and primary mechanism for delivering information to the public, it gets the most requested changes.
What criteria does the IT staff use to evaluate vendors?
To my knowledge, there is no formal criteria established for evaluating vendors. IT will generally evaluate vendors' technical capacity through examination of their portfolio and stated background, to ensure that their skill set aligns with technical requirements. IT will also advise if the stated timeframe and projected costs are within reason.
What internal applications does the IT Department maintain?
What is the server setup?
See here: Servers, AWS Setup, and Backups. The amount of server use depends on the time of year and projected web traffic.
How is your email provided? Are you using Google apps for this, or do you host an Exchange server or something similar?
Email is provided by split delivery: Google apps and qmail (postfix on secondary MX). Email Diagram
What sort of project management tools do you use?
Mantis Bug Tracker is the primary project management tool used by the IT department. Vendors will sometimes use other project management tools with us, including CodeShip.
Also see: NetworkOverview; API and Data Requests
ETL Activity
What sources are the data coming from? How are new data providers added?
Current data sources include:
External Data Providers
Rating, candidate roster, and campaign finance code imports from Research staff
New data providers are added upon request from staff, or if there is an issue with a current data provider.
Is all ETL activity in-bound or is some of it out-bound? If some of it is out-bound, who are the clients of this activity?
Most of the ETL activity is in-bound, apart from an occasional researcher request for a specific dataset. These researchers (clients of the out-bound activity) are most often journalists, academic researchers, and Vote Smart staff. Much of the time, these requests are handled by Research staff.
Which providers and clients are automated and on-going, what of it is one-off?
Demographic data from USPS and Aristotle are imported via a script. The import of Thomas.gov and senate.gov votes are automated within Admin. Rating and candidate roster imports can be one-off for each rating or candidate list, each election. A pyadmin harvester module helps facilitate ratings consumption, including Ratings scrapes by Researchers. Floor speeches are extracted on a daily basis from the GPO to be categorized/tagged by Researchers in a pyadmin module.
Factchecking data is entered by researcher via a pyadmin module
How is this activity performed? Is it a dump-and-load process (some mix of flat files and raw SQL), does it utilize web-based APIs, or is there another approach?
Demographic data is done via bash script with some awk and sql. Finance data are done via python scripts from Research that hit the orgs' APIs or a provided spreadsheet for data, with additional scripts to match to public statements and candidate. Data are imported by IT via a script. Rating imports are done via csv imports + sql in pyadmin. Votes data is taken from .gov sites for the federal level, with manual entry for state-level data.
CategoryITInfo
