2021-07-30T20:17:31,252 Created temporary directory: /tmp/pip-ephem-wheel-cache-fsua3yvx
2021-07-30T20:17:31,258 Created temporary directory: /tmp/pip-req-tracker-8t7_2n67
2021-07-30T20:17:31,259 Initialized build tracking at /tmp/pip-req-tracker-8t7_2n67
2021-07-30T20:17:31,260 Created build tracker: /tmp/pip-req-tracker-8t7_2n67
2021-07-30T20:17:31,260 Entered build tracker: /tmp/pip-req-tracker-8t7_2n67
2021-07-30T20:17:31,261 Created temporary directory: /tmp/pip-wheel-gyb_uymq
2021-07-30T20:17:31,317 WARNING: The index url "/" seems invalid, please provide a scheme.
2021-07-30T20:17:31,318 WARNING: The index url "t" seems invalid, please provide a scheme.
2021-07-30T20:17:31,319 WARNING: The index url "m" seems invalid, please provide a scheme.
2021-07-30T20:17:31,320 WARNING: The index url "p" seems invalid, please provide a scheme.
2021-07-30T20:17:31,320 Looking in indexes: https://pypi.org/simple, /, t, m, p
2021-07-30T20:17:31,323 WARNING: Url '/wrds2pg/' is ignored. It is either a non-existing path or lacks a specific scheme.
2021-07-30T20:17:31,324 WARNING: Url 't/wrds2pg/' is ignored. It is either a non-existing path or lacks a specific scheme.
2021-07-30T20:17:31,324 WARNING: Url 'm/wrds2pg/' is ignored. It is either a non-existing path or lacks a specific scheme.
2021-07-30T20:17:31,325 WARNING: Url 'p/wrds2pg/' is ignored. It is either a non-existing path or lacks a specific scheme.
2021-07-30T20:17:31,327 1 location(s) to search for versions of wrds2pg:
2021-07-30T20:17:31,327 * https://pypi.org/simple/wrds2pg/
2021-07-30T20:17:31,327 Fetching project page and analyzing links: https://pypi.org/simple/wrds2pg/
2021-07-30T20:17:31,329 Getting page https://pypi.org/simple/wrds2pg/
2021-07-30T20:17:31,334 Found index url https://pypi.org/simple
2021-07-30T20:17:31,708   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/36/4d/10bef13832bdc17990beac4a20b47a77976768547aa4bd6137cbafd46f83/wrds2pg-0.1.17-py3-none-any.whl#sha256=3471efec23d7b15c86751d79adc43cb425d7ad46016afc247f1e1992b78083b9 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,711   Found link https://files.pythonhosted.org/packages/ae/9c/ee0b12002821c2c8e21625c915081fc315d9f1ceef100cb330e9116f2d0d/wrds2pg-0.1.17.tar.gz#sha256=665c4fb5c4dcc9f3553eb144053f334ea76e1721b65a96b438fc5a3f78719ef1 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.17
2021-07-30T20:17:31,712   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/ac/24/35ec84d1b765955286a326d0f0c98423730b5818f33c5cdc3e2a2d528adf/wrds2pg-0.1.18-py3-none-any.whl#sha256=896ccb2badb2923b55bb6fbe62a8d74604733c8c9c0181b144e2e2f9aabb629c (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,713   Found link https://files.pythonhosted.org/packages/dd/4f/6c631513b90802ad1fae0c3fb3dbf65efc603d3fee00a1cae4af45f6b2c5/wrds2pg-0.1.18.tar.gz#sha256=7dc2949f1891c602c81f795369c5bd3c72fd257370306eca459e7330e0874988 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.18
2021-07-30T20:17:31,714   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/42/1a/cbdac7035e9b7836ed96c6a9e493b556d3f595245c4f0b239c9ba639d53e/wrds2pg-0.1.20-py3-none-any.whl#sha256=bdca9bdaa2e54c59f2120c519804dc52bf271a35e03a80dd8b1f65f961f28ff6 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,716   Found link https://files.pythonhosted.org/packages/14/81/4c94b875ec21dccdefb6a1846bc4f4d4a6a529a0a6921cbbf580e07f8b6b/wrds2pg-0.1.20.tar.gz#sha256=a9b86e50a278a821f4d1faa3f52312e3e557c332ab234b32def5e10f6afc00ae (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.20
2021-07-30T20:17:31,717   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/e7/a1/192e3bf4b1e122d0b12a094f4ee4fd1667276cb87cbe5d0a53412dc1a9bc/wrds2pg-0.1.21-py3-none-any.whl#sha256=f9ab26b51603e3bf82196a52410a990d95685ee6bedc4a88572340686d09affc (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,719   Found link https://files.pythonhosted.org/packages/4d/4c/5fa9cc2e936e16d0be4a56ce2e8cf03b3ba0f59f49afdebd29f1eda99df4/wrds2pg-0.1.21.tar.gz#sha256=9c3f4e0e21d09ab93555e04c410b5292cf2c89ce2407015de3b289cd943f79ff (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.21
2021-07-30T20:17:31,720   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/4a/52/d9134119b39cf9e939e7cc5ab1317627406af4229fcbdf1085df366e01b4/wrds2pg-0.1.22-py3-none-any.whl#sha256=bfa8fbfc09819fe5524153f82e6fc631069dab5096b0013b03aae5600346a2ab (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,721   Found link https://files.pythonhosted.org/packages/5b/db/ebbf903d32ac113cdd0b3637f1e5dc37a4abf9f63f18dd56835f3611a465/wrds2pg-0.1.22.tar.gz#sha256=9925b6c217bf8c9cf1fa17f6fc08a720b3849b4e95507cf2a905b06536c60ea3 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.22
2021-07-30T20:17:31,722   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/c9/47/a040be2f7d67c120396a27beb1f5e3a6eb97fd24e17cd485ec6d332dad6e/wrds2pg-0.1.23-py3-none-any.whl#sha256=24b5f15364dc798de6d21d3f2d117ce1d7c6e6e0768e8268e7b196943d36d1bd (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,724   Found link https://files.pythonhosted.org/packages/4e/aa/b4f4227ff1d464462f243592bddd638d07e886aa39018ef2bb486624e2f5/wrds2pg-0.1.23.tar.gz#sha256=29557ea7e669dfe1d2f0d4ac9966427698e8369922445e8ca03cc29fd4ff3adb (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.23
2021-07-30T20:17:31,725   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/7b/bf/a713b52aff61a20e5e9d851a69d7c4936d14018ed62759dd78629bfe12cf/wrds2pg-0.1.24-py3-none-any.whl#sha256=8bfd551d615500576f778b977f5771b5011d87a962c1d43e8b7aae7c42f6bafb (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,727   Found link https://files.pythonhosted.org/packages/ca/e9/81095cbac75110e7f53e915ebee06a9925bb32b6d8a307c7a709f9b74dd5/wrds2pg-0.1.24.tar.gz#sha256=b5fdbfe9ab1af360a872257987b354d05998a5e0ee0aefaafe7333787ebc1f16 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 0.1.24
2021-07-30T20:17:31,727   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/01/5d/cfd9a9cfe8194d52e16c24ef053b49225e85f0996c471226fc0cf5522e31/wrds2pg-1.0.0-py3-none-any.whl#sha256=89d17b1aae3ffc78c0c02e87a2f663b83648460da7a8406027d62ae7c9ceb164 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,729   Found link https://files.pythonhosted.org/packages/af/8f/069eed28514231a859da3b110112a0b2078f2c879e3cc5489909982ef2af/wrds2pg-1.0.0.tar.gz#sha256=138924033c17e35fc37dcff72df18398b1152fde1c803f34aa641adc72cb3f66 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.0
2021-07-30T20:17:31,730   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/90/3d/1fc327c68c63a7540a964879a5a124e9104c5a80cd058a5f0659d2945dd5/wrds2pg-1.0.2-py3-none-any.whl#sha256=73be788eb7aee9fce430717615f683c7f912a817238a33697f41f26b85726147 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,732   Found link https://files.pythonhosted.org/packages/fa/b0/49292467a2256c3ecdd7e7fdbd1c4729386bddc06f3a986483a311539f2b/wrds2pg-1.0.2.tar.gz#sha256=13c64ea0b23fd000ccc6f712a448a4c1e49879f9d3d645c58d56cbba9cdc99f8 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.2
2021-07-30T20:17:31,732   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/41/99/2866d67e2fcd10684ccc913eaa896ebcc00c60982133a5adfaa8a39b7aa1/wrds2pg-1.0.3-py3-none-any.whl#sha256=12de805441cbd63af3d66e6a2c651f8f7d15e77dc5c3714035cd450ea1708785 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,734   Found link https://files.pythonhosted.org/packages/0f/bf/b1032fff1645c597e292ac8d588c33bad73ca567c01e52fb149f4aa503d1/wrds2pg-1.0.3.tar.gz#sha256=ebef3b91951fd78a9e3dd51c212c119c8639c03f70aa7c4a273733240fd628fc (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.3
2021-07-30T20:17:31,735   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/0b/6c/bc13811b3a339ec1455a10a03b077d0a7fc7e43b3311201abbfc67d77fde/wrds2pg-1.0.4-py3-none-any.whl#sha256=8197d920152df0686f6cd8ca31af2f46820802334c63a0991ca0ddd1c215c909 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,737   Found link https://files.pythonhosted.org/packages/dc/f4/166d8822216116c22615377ab6493a7c44bd205b8853ae2b0ad51792d9e6/wrds2pg-1.0.4.tar.gz#sha256=e5f9ea4cf47eff9e5062815816f8ae257a9fc0370968f7a6802a0614a8d7a025 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.4
2021-07-30T20:17:31,738   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/78/a7/afce414cb2df44692f2dd5af0e28578c349e337110fb7c73b893cef3c048/wrds2pg-1.0.5-py3-none-any.whl#sha256=21abfd3fabf17eba315d7110979b9ed42f54051f205d859da68568b4699e291b (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,739   Found link https://files.pythonhosted.org/packages/64/a7/18d7edab1e04462bf1aafd2fe6ffc9132f0aa11be6daaf14927c532b73d3/wrds2pg-1.0.5.tar.gz#sha256=bfbd1a931b790209f96b1bd1c45d268ca714b051a65d4d06f51a2bca414674dd (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.5
2021-07-30T20:17:31,740   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/2b/8f/02f345286a6b878468948ab7bc6a56026077f0cbfb25d11440284519ba7a/wrds2pg-1.0.6-py3-none-any.whl#sha256=ba63b46b31e7f735db37bebf8ad301e976085a10628fcb01be40dbc9325267f9 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,742   Found link https://files.pythonhosted.org/packages/8c/06/abb4c39f3e3f730fffe349e03e1172e31c95fbb9edf4c9257bfb0b6e2c62/wrds2pg-1.0.6.tar.gz#sha256=33efcef845f9d47adb892052a2865ecb6dafb9541ead49a97840a0659d76ca91 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.6
2021-07-30T20:17:31,743   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/3f/b5/99b90fee6aaf0b150803cfbd6d86e0052d292257dd101e803df4d945ad7b/wrds2pg-1.0.8-py3-none-any.whl#sha256=4bca52110d40065aada2ca48ed090ab85a34add55c09c804f9586fd454ae3205 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,744   Found link https://files.pythonhosted.org/packages/eb/9f/a506b309ee9299d4d9cd3f5c1407e05da30c09cbbe8acf67e60de11a1b4e/wrds2pg-1.0.8.tar.gz#sha256=d5b5e5e738d01c7509660b839c8a949ff5210a81eedd2b514f8bdfa044752e96 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.8
2021-07-30T20:17:31,745   Skipping link: No binaries permitted for wrds2pg: https://files.pythonhosted.org/packages/80/6d/34291718af062c4c329fbaeaaa582e011da15ce1193364f668581c3bd983/wrds2pg-1.0.9-py3-none-any.whl#sha256=31c6a2598fe8d8b97d73bcad19025da0249d8c835873e8ae12504bb585268cd0 (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3)
2021-07-30T20:17:31,747   Found link https://files.pythonhosted.org/packages/27/20/2b9788aef9e54ea3d590a456caa294d1c177fd05029e8f70b99022369f68/wrds2pg-1.0.9.tar.gz#sha256=abb716e343d49a455f949c7c87c1a08a0748f1233917d4b9282df55245bdca6c (from https://pypi.org/simple/wrds2pg/) (requires-python:>=3), version: 1.0.9
2021-07-30T20:17:31,781 Given no hashes to check 1 links for project 'wrds2pg': discarding no candidates
2021-07-30T20:17:31,811 Collecting wrds2pg==1.0.9
2021-07-30T20:17:31,814   Created temporary directory: /tmp/pip-unpack-90n62ywe
2021-07-30T20:17:31,896   Downloading wrds2pg-1.0.9.tar.gz (11 kB)
2021-07-30T20:17:31,984   Added wrds2pg==1.0.9 from https://files.pythonhosted.org/packages/27/20/2b9788aef9e54ea3d590a456caa294d1c177fd05029e8f70b99022369f68/wrds2pg-1.0.9.tar.gz#sha256=abb716e343d49a455f949c7c87c1a08a0748f1233917d4b9282df55245bdca6c to build tracker '/tmp/pip-req-tracker-8t7_2n67'
2021-07-30T20:17:31,986     Running setup.py (path:/tmp/pip-wheel-gyb_uymq/wrds2pg_ba71131169ad4c16a113ae652dc59bb8/setup.py) egg_info for package wrds2pg
2021-07-30T20:17:31,988     Created temporary directory: /tmp/pip-pip-egg-info-w0yfphqm
2021-07-30T20:17:31,988     Running command python setup.py egg_info
2021-07-30T20:17:33,799     # WRDS (or SAS) to PostgreSQL
2021-07-30T20:17:33,800     This software has two functions:
2021-07-30T20:17:33,801     - Download tables from [WRDS](https://wrds-web.wharton.upenn.edu/wrds/) and feed them to a PostgreSQL database. (Requires access to WRDS and to the data in question.)
2021-07-30T20:17:33,801     - Import a SAS file (`*.sas7dbat`) into a PostgreSQL database.

2021-07-30T20:17:33,803     ## Requirements

2021-07-30T20:17:33,804     ### 1. Python
2021-07-30T20:17:33,805     The software uses Python 3 and depends on Pandas, SQLAlchemy and Paramiko. In addition, the Python scripts generally interact with PostgreSQL using SQLAlchemy and the [Psycopg](http://initd.org/psycopg/) library.
2021-07-30T20:17:33,805     These dependencies are installed when you use Pip (see instructions below).

2021-07-30T20:17:33,807     ### 2. A WRDS ID
2021-07-30T20:17:33,807     To use public-key authentication to access WRDS, follow hints taken from [here](https://debian-administration.org/article/152/Password-less_logins_with_OpenSSH) to set up a public key.
2021-07-30T20:17:33,808     Copy that key to the WRDS server from the terminal on your computer.
2021-07-30T20:17:33,809     (Note that this code assumes you have a directory `.ssh` in your home directory. If not, log into WRDS via SSH, then type `mkdir ~/.ssh` to create this.)
2021-07-30T20:17:33,810     Here's code (for me) to create the key and send it to WRDS:
2021-07-30T20:17:33,810     ```
2021-07-30T20:17:33,811     ssh-keygen -t rsa
2021-07-30T20:17:33,811     cat ~/.ssh/id_rsa.pub | ssh iangow@wrds-cloud.wharton.upenn.edu "cat >> ~/.ssh/authorized_keys"
2021-07-30T20:17:33,812     ```
2021-07-30T20:17:33,813     Use an empty passphrase in setting up the key so that the scripts can run without user intervention.

2021-07-30T20:17:33,814     ### 3. PostgreSQL
2021-07-30T20:17:33,815     You should have a PostgreSQL database to store the data.

2021-07-30T20:17:33,816     ### 4. Environment variables

2021-07-30T20:17:33,817     Environment variables that the code can use include:

2021-07-30T20:17:33,819     - `PGDATABASE`: The name of the PostgreSQL database you use.
2021-07-30T20:17:33,819     - `PGUSER`: Your username on the PostgreSQL database.
2021-07-30T20:17:33,820     - `PGHOST`: Where the PostgreSQL database is to be found (this will be `localhost` if its on the same machine as you're running the code on)
2021-07-30T20:17:33,820     - `WRDS_ID`: Your [WRDS](https://wrds-web.wharton.upenn.edu/wrds/) ID.

2021-07-30T20:17:33,822     I set these environment variables in `~/.profile`:

2021-07-30T20:17:33,823     ```
2021-07-30T20:17:33,823     export PGHOST="localhost"
2021-07-30T20:17:33,824     export PGDATABASE="crsp"
2021-07-30T20:17:33,825     export WRDS_ID="iangow"
2021-07-30T20:17:33,825     export PGUSER="igow"
2021-07-30T20:17:33,826     ```

2021-07-30T20:17:33,827     ## Using the function `wrds_update`.

2021-07-30T20:17:33,829     Two arguments `table_name` and `schema` are required.

2021-07-30T20:17:33,830     ### 1. WRDS Settings
2021-07-30T20:17:33,830     Set `WRDS_ID`  using either `wrds_id=your_wrds_id` in the function call or the environment variable `WRDS_ID`.

2021-07-30T20:17:33,832     ### 2. Environment variables
2021-07-30T20:17:33,832     The software will use the environment variables `PGHOST`, `PGDATABASE`, and `PGUSER` if you If you have set them. Otherwise, you need to provide values as arguments to `wrds_udpate()`. Default `PGPORT` is`5432`.

2021-07-30T20:17:33,834     ### 3. Table settings
2021-07-30T20:17:33,834     To tailor your request, specify the following arguments:

2021-07-30T20:17:33,836     - `fix_missing`: set to `True` to fix missing values. This addresses special missign values, which SAS's `PROC EXPORT` dumps as strings. Default value is `False`.
2021-07-30T20:17:33,836     - `fix_cr`: set to `True` to fix characters. Default value is `False`.
2021-07-30T20:17:33,837     - `drop`: add column names to be dropped (e.g., `drop="id name"` will drop columns `id` and `name`).
2021-07-30T20:17:33,838     - `obs`: add maxium number of observations (e.g., `obs=10` will import the first 10 rows from the table on WRDS).
2021-07-30T20:17:33,838     - `rename`: rename columns (e.g., `rename="fee=mngt_fee"` renames `fee` to `mngt_fee`).
2021-07-30T20:17:33,839     - `force`: set to `True` to force update. Default value is `False`.

2021-07-30T20:17:33,840     ## Importing SAS data into PostgreSQL
2021-07-30T20:17:33,840     The software can also upload SAS file directly to PostgreSQL.
2021-07-30T20:17:33,841     You need to have local SAS in order to use this function.
2021-07-30T20:17:33,842     Use `fpath` to specify the path to the file to be imported

2021-07-30T20:17:33,843     ### Examples
2021-07-30T20:17:33,843     Here are some examples.

2021-07-30T20:17:33,845     If you are at the home directory of this git repo, you can import and use the software as shown below.

2021-07-30T20:17:33,846     To install it from Github:

2021-07-30T20:17:33,848     ```
2021-07-30T20:17:33,848     sudo -H pip3 install git+https://github.com/iangow/wrds2pg --upgrade
2021-07-30T20:17:33,849     ```

2021-07-30T20:17:33,851     This software is also available from PyPI. To install it from [PyPI](https://pypi.org/project/wrds2pg/):
2021-07-30T20:17:33,851     ```
2021-07-30T20:17:33,852     pip3 install wrds2pg
2021-07-30T20:17:33,853     ```
2021-07-30T20:17:33,854     Example usage:
2021-07-30T20:17:33,855     ```py
2021-07-30T20:17:33,855     from wrds2pg import wrds_update

2021-07-30T20:17:33,857     # 1. Download crsp.mcti from wrds and upload to pg as crps.mcti
2021-07-30T20:17:33,858     # Simplest version
2021-07-30T20:17:33,858     wrds_update(table_name="mcti", schema="crsp")

2021-07-30T20:17:33,860     # Tailored arguments
2021-07-30T20:17:33,861     wrds_update(table_name="mcti", schema="crsp", host=your_pghost,
2021-07-30T20:17:33,862     	dbname=your_pg_database,
2021-07-30T20:17:33,862     	fix_missing=True, fix_cr=True, drop="b30ret b30ind", obs=10,
2021-07-30T20:17:33,863     	rename="caldt=calendar_date", force=True)

2021-07-30T20:17:33,864     # 2. Upload test.sas7dbat to pg as crsp.mcti
2021-07-30T20:17:33,865     wrds_update(table_name="mcti", schema="crsp", fpath="your_path/test.sas7dbat")
2021-07-30T20:17:33,866     ```

2021-07-30T20:17:33,866     ### Report bugs
2021-07-30T20:17:33,867     Author: Ian Gow, <iandgow@gmail.com>
2021-07-30T20:17:33,867     Contributor: Jingyu Zhang, <jingyu.zhang@chicagobooth.edu>

2021-07-30T20:17:33,869     running egg_info
2021-07-30T20:17:33,870     creating /tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info
2021-07-30T20:17:33,870     writing top-level names to /tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/top_level.txt
2021-07-30T20:17:33,871     writing /tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/PKG-INFO
2021-07-30T20:17:33,872     writing requirements to /tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/requires.txt
2021-07-30T20:17:33,872     writing dependency_links to /tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/dependency_links.txt
2021-07-30T20:17:33,873     writing manifest file '/tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/SOURCES.txt'
2021-07-30T20:17:34,215     reading manifest file '/tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/SOURCES.txt'
2021-07-30T20:17:34,222     writing manifest file '/tmp/pip-pip-egg-info-w0yfphqm/wrds2pg.egg-info/SOURCES.txt'
2021-07-30T20:17:34,357   Source in /tmp/pip-wheel-gyb_uymq/wrds2pg_ba71131169ad4c16a113ae652dc59bb8 has version 1.0.9, which satisfies requirement wrds2pg==1.0.9 from https://files.pythonhosted.org/packages/27/20/2b9788aef9e54ea3d590a456caa294d1c177fd05029e8f70b99022369f68/wrds2pg-1.0.9.tar.gz#sha256=abb716e343d49a455f949c7c87c1a08a0748f1233917d4b9282df55245bdca6c
2021-07-30T20:17:34,358   Removed wrds2pg==1.0.9 from https://files.pythonhosted.org/packages/27/20/2b9788aef9e54ea3d590a456caa294d1c177fd05029e8f70b99022369f68/wrds2pg-1.0.9.tar.gz#sha256=abb716e343d49a455f949c7c87c1a08a0748f1233917d4b9282df55245bdca6c from build tracker '/tmp/pip-req-tracker-8t7_2n67'
2021-07-30T20:17:34,372 Created temporary directory: /tmp/pip-unpack-ua5752a0
2021-07-30T20:17:34,373 Building wheels for collected packages: wrds2pg
2021-07-30T20:17:34,381   Created temporary directory: /tmp/pip-wheel-n22p65x8
2021-07-30T20:17:34,383   Building wheel for wrds2pg (setup.py): started
2021-07-30T20:17:34,383   Destination directory: /tmp/pip-wheel-n22p65x8
2021-07-30T20:17:34,384   Running command /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-wheel-gyb_uymq/wrds2pg_ba71131169ad4c16a113ae652dc59bb8/setup.py'"'"'; __file__='"'"'/tmp/pip-wheel-gyb_uymq/wrds2pg_ba71131169ad4c16a113ae652dc59bb8/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-n22p65x8
2021-07-30T20:17:35,870   # WRDS (or SAS) to PostgreSQL
2021-07-30T20:17:35,871   This software has two functions:
2021-07-30T20:17:35,872   - Download tables from [WRDS](https://wrds-web.wharton.upenn.edu/wrds/) and feed them to a PostgreSQL database. (Requires access to WRDS and to the data in question.)
2021-07-30T20:17:35,872   - Import a SAS file (`*.sas7dbat`) into a PostgreSQL database.

2021-07-30T20:17:35,874   ## Requirements

2021-07-30T20:17:35,875   ### 1. Python
2021-07-30T20:17:35,876   The software uses Python 3 and depends on Pandas, SQLAlchemy and Paramiko. In addition, the Python scripts generally interact with PostgreSQL using SQLAlchemy and the [Psycopg](http://initd.org/psycopg/) library.
2021-07-30T20:17:35,876   These dependencies are installed when you use Pip (see instructions below).

2021-07-30T20:17:35,878   ### 2. A WRDS ID
2021-07-30T20:17:35,878   To use public-key authentication to access WRDS, follow hints taken from [here](https://debian-administration.org/article/152/Password-less_logins_with_OpenSSH) to set up a public key.
2021-07-30T20:17:35,879   Copy that key to the WRDS server from the terminal on your computer.
2021-07-30T20:17:35,880   (Note that this code assumes you have a directory `.ssh` in your home directory. If not, log into WRDS via SSH, then type `mkdir ~/.ssh` to create this.)
2021-07-30T20:17:35,880   Here's code (for me) to create the key and send it to WRDS:
2021-07-30T20:17:35,881   ```
2021-07-30T20:17:35,881   ssh-keygen -t rsa
2021-07-30T20:17:35,881   cat ~/.ssh/id_rsa.pub | ssh iangow@wrds-cloud.wharton.upenn.edu "cat >> ~/.ssh/authorized_keys"
2021-07-30T20:17:35,882   ```
2021-07-30T20:17:35,882   Use an empty passphrase in setting up the key so that the scripts can run without user intervention.

2021-07-30T20:17:35,883   ### 3. PostgreSQL
2021-07-30T20:17:35,884   You should have a PostgreSQL database to store the data.

2021-07-30T20:17:35,885   ### 4. Environment variables

2021-07-30T20:17:35,886   Environment variables that the code can use include:

2021-07-30T20:17:35,887   - `PGDATABASE`: The name of the PostgreSQL database you use.
2021-07-30T20:17:35,888   - `PGUSER`: Your username on the PostgreSQL database.
2021-07-30T20:17:35,888   - `PGHOST`: Where the PostgreSQL database is to be found (this will be `localhost` if its on the same machine as you're running the code on)
2021-07-30T20:17:35,889   - `WRDS_ID`: Your [WRDS](https://wrds-web.wharton.upenn.edu/wrds/) ID.

2021-07-30T20:17:35,890   I set these environment variables in `~/.profile`:

2021-07-30T20:17:35,891   ```
2021-07-30T20:17:35,892   export PGHOST="localhost"
2021-07-30T20:17:35,893   export PGDATABASE="crsp"
2021-07-30T20:17:35,893   export WRDS_ID="iangow"
2021-07-30T20:17:35,894   export PGUSER="igow"
2021-07-30T20:17:35,894   ```

2021-07-30T20:17:35,896   ## Using the function `wrds_update`.

2021-07-30T20:17:35,897   Two arguments `table_name` and `schema` are required.

2021-07-30T20:17:35,898   ### 1. WRDS Settings
2021-07-30T20:17:35,899   Set `WRDS_ID`  using either `wrds_id=your_wrds_id` in the function call or the environment variable `WRDS_ID`.

2021-07-30T20:17:35,900   ### 2. Environment variables
2021-07-30T20:17:35,901   The software will use the environment variables `PGHOST`, `PGDATABASE`, and `PGUSER` if you If you have set them. Otherwise, you need to provide values as arguments to `wrds_udpate()`. Default `PGPORT` is`5432`.

2021-07-30T20:17:35,902   ### 3. Table settings
2021-07-30T20:17:35,903   To tailor your request, specify the following arguments:

2021-07-30T20:17:35,904   - `fix_missing`: set to `True` to fix missing values. This addresses special missign values, which SAS's `PROC EXPORT` dumps as strings. Default value is `False`.
2021-07-30T20:17:35,904   - `fix_cr`: set to `True` to fix characters. Default value is `False`.
2021-07-30T20:17:35,905   - `drop`: add column names to be dropped (e.g., `drop="id name"` will drop columns `id` and `name`).
2021-07-30T20:17:35,906   - `obs`: add maxium number of observations (e.g., `obs=10` will import the first 10 rows from the table on WRDS).
2021-07-30T20:17:35,906   - `rename`: rename columns (e.g., `rename="fee=mngt_fee"` renames `fee` to `mngt_fee`).
2021-07-30T20:17:35,907   - `force`: set to `True` to force update. Default value is `False`.

2021-07-30T20:17:35,908   ## Importing SAS data into PostgreSQL
2021-07-30T20:17:35,909   The software can also upload SAS file directly to PostgreSQL.
2021-07-30T20:17:35,909   You need to have local SAS in order to use this function.
2021-07-30T20:17:35,910   Use `fpath` to specify the path to the file to be imported

2021-07-30T20:17:35,911   ### Examples
2021-07-30T20:17:35,912   Here are some examples.

2021-07-30T20:17:35,913   If you are at the home directory of this git repo, you can import and use the software as shown below.

2021-07-30T20:17:35,914   To install it from Github:

2021-07-30T20:17:35,916   ```
2021-07-30T20:17:35,916   sudo -H pip3 install git+https://github.com/iangow/wrds2pg --upgrade
2021-07-30T20:17:35,917   ```

2021-07-30T20:17:35,918   This software is also available from PyPI. To install it from [PyPI](https://pypi.org/project/wrds2pg/):
2021-07-30T20:17:35,919   ```
2021-07-30T20:17:35,919   pip3 install wrds2pg
2021-07-30T20:17:35,920   ```
2021-07-30T20:17:35,920   Example usage:
2021-07-30T20:17:35,921   ```py
2021-07-30T20:17:35,922   from wrds2pg import wrds_update

2021-07-30T20:17:35,923   # 1. Download crsp.mcti from wrds and upload to pg as crps.mcti
2021-07-30T20:17:35,923   # Simplest version
2021-07-30T20:17:35,924   wrds_update(table_name="mcti", schema="crsp")

2021-07-30T20:17:35,925   # Tailored arguments
2021-07-30T20:17:35,926   wrds_update(table_name="mcti", schema="crsp", host=your_pghost,
2021-07-30T20:17:35,927   	dbname=your_pg_database,
2021-07-30T20:17:35,927   	fix_missing=True, fix_cr=True, drop="b30ret b30ind", obs=10,
2021-07-30T20:17:35,928   	rename="caldt=calendar_date", force=True)

2021-07-30T20:17:35,929   # 2. Upload test.sas7dbat to pg as crsp.mcti
2021-07-30T20:17:35,929   wrds_update(table_name="mcti", schema="crsp", fpath="your_path/test.sas7dbat")
2021-07-30T20:17:35,930   ```

2021-07-30T20:17:35,931   ### Report bugs
2021-07-30T20:17:35,932   Author: Ian Gow, <iandgow@gmail.com>
2021-07-30T20:17:35,932   Contributor: Jingyu Zhang, <jingyu.zhang@chicagobooth.edu>

2021-07-30T20:17:36,262   running bdist_wheel
2021-07-30T20:17:36,276   running build
2021-07-30T20:17:36,277   running build_py
2021-07-30T20:17:36,613   creating build
2021-07-30T20:17:36,614   creating build/lib
2021-07-30T20:17:36,615   creating build/lib/wrds2pg
2021-07-30T20:17:36,618   copying wrds2pg/wrds2pg.py -> build/lib/wrds2pg
2021-07-30T20:17:36,624   copying wrds2pg/__init__.py -> build/lib/wrds2pg
2021-07-30T20:17:36,634   installing to build/bdist.linux-armv7l/wheel
2021-07-30T20:17:36,635   running install
2021-07-30T20:17:36,637   running install_lib
2021-07-30T20:17:36,643   creating build/bdist.linux-armv7l
2021-07-30T20:17:36,644   creating build/bdist.linux-armv7l/wheel
2021-07-30T20:17:36,647   creating build/bdist.linux-armv7l/wheel/wrds2pg
2021-07-30T20:17:36,649   copying build/lib/wrds2pg/wrds2pg.py -> build/bdist.linux-armv7l/wheel/wrds2pg
2021-07-30T20:17:36,656   copying build/lib/wrds2pg/__init__.py -> build/bdist.linux-armv7l/wheel/wrds2pg
2021-07-30T20:17:36,659   running install_egg_info
2021-07-30T20:17:36,700   running egg_info
2021-07-30T20:17:36,702   writing wrds2pg.egg-info/PKG-INFO
2021-07-30T20:17:36,707   writing top-level names to wrds2pg.egg-info/top_level.txt
2021-07-30T20:17:36,709   writing dependency_links to wrds2pg.egg-info/dependency_links.txt
2021-07-30T20:17:36,711   writing requirements to wrds2pg.egg-info/requires.txt
2021-07-30T20:17:36,720   reading manifest file 'wrds2pg.egg-info/SOURCES.txt'
2021-07-30T20:17:36,728   writing manifest file 'wrds2pg.egg-info/SOURCES.txt'
2021-07-30T20:17:36,731   Copying wrds2pg.egg-info to build/bdist.linux-armv7l/wheel/wrds2pg-1.0.9-py3.5.egg-info
2021-07-30T20:17:36,754   running install_scripts
2021-07-30T20:17:36,948   creating build/bdist.linux-armv7l/wheel/wrds2pg-1.0.9.dist-info/WHEEL
2021-07-30T20:17:37,146   Building wheel for wrds2pg (setup.py): finished with status 'done'
2021-07-30T20:17:37,151   Created wheel for wrds2pg: filename=wrds2pg-1.0.9-py3-none-any.whl size=11529 sha256=544627010f3f3125f38a8b442475cb841a1e7b63e9644d7ed1c98245e5e8ffeb
2021-07-30T20:17:37,151   Stored in directory: /tmp/pip-ephem-wheel-cache-fsua3yvx/wheels/0c/a1/8b/7253e6cb4fcd0adf8a969c1aa0f72e5aa8589c9ee6639d0d68
2021-07-30T20:17:37,169 Successfully built wrds2pg
2021-07-30T20:17:37,175 Removed build tracker: '/tmp/pip-req-tracker-8t7_2n67'