Blog

Creating a simple python package

2017-06-27

I signed up for GitHub after finishing my PhD in 2016. One of the first repositories I set up was a port of some of my MATLAB code for generating fractional Brownian motion realizations to python. It was basically just a README.md citing the sources of the algorithms used and a single file called fbm.py.

A few months ago I noticed the repository had picked up a few stars and was being cloned a couple times each week. I figured that if people were interested in this code that I should try to refactor it and create an actual python package that anyone could download and implement using pip. So that’s what I did. The package is called fbm and is available on pypi.

I’ll go through the steps I took here. After refactoring the code and adding some nice functionality, the package directory structure looked like this. A single module with a small set of unit tests in the tests folder.

packagename/
    packagename/
        __init__.py
        packagename.py
    tests/
        packagename_tests.py
    requirements.txt
    README.rst
    .gitignore

setup.py

The first step is to create a file called setup.py in the top level directory of the project which imports the setup function from the setuptools module. The setuptools package should always be used in favor of distutils.

from setuptools import setup


def readme():
    with open('README.rst') as f:
        return f.read()

setup(name='packagename',
      version='0.1.0',
      description='A simple module',
      long_description=readme(),
      license='MIT',
      author='author name',
      author_email='author@email.com',
      url='github_url',
      packages=['packagename'],
      zip_safe=False,
)

This is a bare minimum setup.py. A few notes:

name should be a package name that is not already taken. You can easily search pypi to see if the package name you want is already taken.
version should be selected based on the [Semantic Versioning](Semantic Versioning) guide. It’s not required but it’s a good practice.
description should be a few words describing the package.
long_description can be a more detailed description. In this example you can see we read in the README.rst file. Note that pypi will process rst (ReStructuredText) files and import them to the page that is created for the package.
license is the license you want to use for your software. If you are unsure, you should check with choosealicense.
author_name is your name.
author_email is your email (must be included if you provide your name).
url is a link to your project. I used the repository url.
packages can me a manual list of packages or you can use the find_packages function of setuptools.
zip_safe see here. Set it to False if you’re unsure.

There are a number of additional arguments you can specify here. You should use install_requires if your package has any dependencies on other packages. If you want to include the README, LICENSE, or other non-python files you should also use the include_package_data argument as True. You can also use the classifiers argument to specify categorical tags to your package.

If include_package_data is True, you’ll need to create a MANIFEST.in file in the top level of your project. Inside the file you’ll want to include any additional relevant files to your package, for example:

include README.rst
include LICENSE.txt

You’ll also want to modify your .gitignore file to ignore some files that will be generated when you build your package. Here is what mine looks like. The .env file is for working in a virtual environment. The .tox file is for the tox package which automates virtual-env based testing.

# Compiled python modules.
*.pyc
# Setuptools distribution folder.
/dist/
# Python egg metadata, regenerated from source files by setuptools.
/*.egg-info
# Python virtualenv .env file
.env
# tox testing
.tox/
# packaging
MANIFEST
build/

Put altogether your package directory might look something like this.

packagename/
    packagename/
        __init__.py
        packagename.py
    tests/
        packagename_tests.py
    requirements.txt
    README.rst
    LICENSE.txt
    MANIFEST.in
    setup.py
    .gitignore

Creating the distribution

To create a source distribution of your project run the command.

python setup.py sdist

If your project is pure python you can create a universal wheel distribution.

python setup.py bdist_wheel

Both of these commands will create .tar.gz and .whl files, respectively, in the dist/ folder.

Uploading to pypi

To get your package on pypi, you’ll need to register an account. You should create an account on pypi and the testpypi, which will allow you to test your upload before putting the real thing up on pypi. You should also install the twine package using the command pip install twine.

Next you should create and open the file ~/.pypirc in a text editor. It should look like this with your pypi and testpypi credentials written in:

[distutils]
index-servers =
    pypi
    testpypi

[pypi]
repository=https://pypi.python.org/pypi
username=pypi_username
password=pypi_password

[testpypi]
repository=https://test.pypi.org/legacy/
username=testpypi_username
password=testpypi_password

You should be extremely careful with this file as you are storing your personal credentials here in plaintext.

We’ll use testpypi to test our package upload. We’ll have to register first on the test server. To register the project on testpypi:

python setup.py register -r https://testpypi.python.org/pypi

To upload:

twine upload dist/packagename-version.tar.gz -r testpypi
twine upload dist/packagename-version-py2.py3-none-any.whl -r testpypi

You should see a page has been created at https://testpypi.python.org/pypi/packagename/. To test that it worked:

pip install -i https://testpypi.python.org/pypi packagename

If everything worked, you can run this last command to upload your package to the actual pypi (handles registration automatically), replacing the packagename and version with the proper values:

twine upload dist/packagename-version.tar.gz
twine upload dist/packagename-version-py2.py3-none-any.whl

You’re done and you should now be able to see your package on pypi!

Updating your package

If you plan to maintain your project you’ll want to upload a new version when it becomes ready. Remember to increase the version number in setup.py before building the new distributions. Once the new source distribution and wheel files are ready, just run:

twine upload dist/packagename-newversion.tar.gz
twine upload dist/packagename-newversion-py2.py3-none-any.whl

And remember it’s always a good idea to use the testpypi first.

flynn.gg