Metadata-Version: 2.1
Name: bandersnatch
Version: 3.1.3
Summary: Mirroring tool that implements the client (mirror) side of PEP 381
Home-page: https://github.com/pypa/bandersnatch/
Author: Christian Theune
Author-email: ct@flyingcircus.io
License: Academic Free License, version 3
Description: [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
        [![Build Status](https://travis-ci.org/pypa/bandersnatch.svg?branch=master)](https://travis-ci.org/pypa/bandersnatch)
        [![Documentation Status](https://readthedocs.org/projects/bandersnatch/badge/?version=latest)](http://bandersnatch.readthedocs.io/en/latest/?badge=latest)
        [![Updates](https://pyup.io/repos/github/pypa/bandersnatch/shield.svg)](https://pyup.io/repos/github/pypa/bandersnatch/)
        [![Downloads](https://pepy.tech/badge/bandersnatch)](https://pepy.tech/project/bandersnatch)
        
        ----
        
        This is a PyPI mirror client according to `PEP 381`
        http://www.python.org/dev/peps/pep-0381/.
        
        ## Installation
        
        The following instructions will place the bandersnatch executable in a
        virtualenv under `bandersnatch/bin/bandersnatch`.
        
        - bandersnatch **requires** `>= Python 3.6.1`
        
        ### pip
        
        This installs the latest stable, released version.
        
        ```
          $ python3.6 -m venv bandersnatch
          $ bandersnatch/bin/pip install bandersnatch
        ```
        
        ## Quickstart
        
        * Run ``bandersnatch mirror`` - it will create an empty configuration file
          for you in ``/etc/bandersnatch.conf``.
        * Review ``/etc/bandersnatch.conf`` and adapt to your needs.
        * Run ``bandersnatch mirror`` again. It will populate your mirror with the
          current status of all PyPI packages.
          Current mirror package size can be seen here: https://pypi.org/stats/
        * A ``blacklist`` or ``whitelist`` can be created to cut down your mirror size.
          Example blacklist generation tool: https://github.com/cooperlees/pypistats
        * Run ``bandersnatch mirror`` regularly to update your mirror with any
          intermediate changes.
        
        ### Webserver
        
        Configure your webserver to serve the ``web/`` sub-directory of the mirror.
        For nginx it should look something like this::
        
        ```
            server {
                listen 127.0.0.1:80;
                server_name <mymirrorname>;
                root <path-to-mirror>/web;
                autoindex on;
                charset utf-8;
            }
        ```
        
        * Note that it is a good idea to have your webserver publish the HTML index
          files correctly with UTF-8 as the charset. The index pages will work without
          it but if humans look at the pages the characters will end up looking funny.
        
        * Make sure that the webserver uses UTF-8 to look up unicode path names. nginx
          gets this right by default - not sure about others.
        
        
        ### Cron jobs
        
        You need to set up one cron job to run the mirror itself.
        
        Here's a sample that you could place in `/etc/cron.d/bandersnatch`:
        
        ```
            LC_ALL=en_US.utf8
            */2 * * * * root bandersnatch mirror |& logger -t bandersnatch[mirror]
        ```
        
        This assumes that you have a ``logger`` utility installed that will convert the
        output of the commands to syslog entries.
        
        
        ### Maintenance
        
        bandersnatch does not keep much local state in addition to the mirrored data.
        In general you can just keep rerunning `bandersnatch mirror` to make it fix
        errors.
        
        If you want to force bandersnatch to check everything against the master PyPI::
        
        * run `bandersnatch mirror --force-check` to move status files if they exist in your mirror directory in order get a full sync.
        
        Be aware, that full syncs likely take hours depending on PyPIs performance and
        your network latency and bandwidth.
        
        ### Operational notes
        
        #### Case-sensitive filesystem needed
        
        You need to run bandersnatch on a case-sensitive filesystem.
        
        OS X natively does this OK even though the filesystem is not strictly
        case-sensitive and bandersnatch will work fine when running on OS X. However,
        tarring a bandersnatch data directory and moving it to, e.g. Linux with a
        case-sensitive filesystem will lead to inconsistencies. You can fix those by
        deleting the status files and have bandersnatch run a full check on your data.
        
        #### Many sub-directories needed
        
        The PyPI has a quite extensive list of packages that we need to maintain in a
        flat directory. Filesystems with small limits on the number of sub-directories
        per directory can run into a problem like this::
        
          2013-07-09 16:11:33,331 ERROR: Error syncing package: zweb@802449
          OSError: [Errno 31] Too many links: '../pypi/web/simple/zweb'
        
        Specifically we recommend to avoid using ext3. Ext4 and newer does not have the
        limitation of 32k sub-directories.
        
        #### Client Compatibility
        
        A bandersnatch static mirror is compatible only to the "static",  cacheable
        parts of PyPI that are needed to support package installation. It does not
        support more dynamic APIs of PyPI that maybe be used by various clients for
        other purposes.
        
        An example of an unsupported API is PyPI's XML-RPC interface, which is used
        when running `pip search`.
        
        ### zc.buildout
        - *WARNING* This feature has not been kept updated
        
        This installs the current development version. Use `git checkout <commit-hash>` and run buildout again to choose a specific release.
        
        ```
          $ git clone https://github.com/pypa/bandersnatch
          $ cd bandersnatch
          $ ./bootstrap.sh
        ```
        
        ### Contact
        
        If you have questions or comments, please submit a bug report to
        https://github.com/pypa/bandersnatch/issues/new
        
        ### Code of Conduct
        
        Everyone interacting in the bandersnatch project's codebases, issue trackers,
        chat rooms, and mailing lists is expected to follow the
        [PyPA Code of Conduct](https://www.pypa.io/en/latest/code-of-conduct/).
        
        
        ### Kudos
        
        This client is based on the original pep381client by *Martin v. Loewis*.
        
        *Richard Jones* was very patient answering questions at PyCon 2013 and made the
        protocol more reliable by implementing some PyPI enhancements.
        
        *Christian Theune* for creating and maintaining `bandersnatch` for many years!
        
        
        ## 3.1.3 (2018-12-26)
        
        - Print help message when no arguments given to bandersnatch - Thanks **@GreatBahram**
        - aiohttp >= 3.5.0 test and we no longer have `.netrc` error message
        
        ## 3.1.2 (2018-12-02)
        
        - Load default config or passed in config file only *(not both)* - `Fixes #95` - Thanks **@GreatBahram**
        - Add `--force-check` to mirror to enable full PyPI Syncs - `Fixes #97` - Thanks **@GreatBahram**
        
        ## 3.1.1 (2018-11-25)
        
        - Add missing `filelock` dependency to `setup.py` `Fixes #93`
        
        ## 3.1.0 (2018-11-25)
        
        - Store N versions of index.html - `Fixes #9` - Thanks **@yeraydiazdiaz**
        - Add CI Integration test - `Fixes #78` - Thanks **@cooperlees**
        - Test / pin to latest dependencies via PyUP - `Fixes #70` - Thanks **@cooperlees**
        - Revert pinning versions in `setup.py` - `Fixes #81`
        - Add Pre-release + regex filter plguins `Fixes #83` - Thanks **@yeraydiazdiaz**
        
        ## 3.0.1 (2018-10-30)
        
        - Fix setup.py *url* to point at GitHub (https://github.com/pypa/bandersnatch)
        
        # 3.0.0 (2018-10-30)
        
        - Move to asyncio executors around request calls `Fixes #81` *(on BitBucket)*
        - Use platform.uname() to support Windows `Fixes #19`
        - Add **bandersnatch verify** subcommand to re-download + delete unneeded packages `Fixes #8` + many follow on Issues during testing - Thanks **electricworry** & **tau3** for testing + fixes!
        - Introduce much more Lint checks (black, isort, mypy) other than flake8 - Thanks **@asottile**
        - Make tox run lint checks + print out test coverage - Thanks **@cooperlees**
        - Add whitelist + blacklist plugins - Thanks **@dwighthubbard**
        - Add generated documentation - Thanks **@dwighthubbard**
        - Move to requiring Python >= 3.6.1 `Fixes #66`
        
        **Moved to GitHub @ PyCon US 2018 - All `Fixes` now refer to GitHub issues**
        
        ## 2.2.1 (2018-05-01)
        
        - Fix missed MANIFEST.in change for this file :P `Fixes #108` - Thanks **@cooperlees**
        
        ## 2.2.0 (2018-03-28)
        
        - Allow digest_name to be specified. `Fixes #105` - Thanks **@ewdurbin** !
        - synchronize generated index pages with warehouse - Thanks **@ewdurbin** !
        - Allow root_uri to be configured - Thanks **@ewdurbin** !
        -- This is how warehouse (pypi.org) will function
        
        
        ## 2.1.3 (2018-03-04)
        
        - Change version from using pkg_resources and set it in package __init__.py.
          `Fixes #98`.
        - Add ability to blacklist packages to sync via conf file. `Fixes #100`.
        
        
        ## 2.1.2
        
        - Add saving of JSON metadata grabbed from pypi.facebook.com for syncing `Fixes #91` - Thanks **@cooperlees**
        -- Can be disabled via config and disabled by default
        -- bandersnatch symlinks WEB_ROOT/pypi/PKG_NAME/json to WEB_ROOT/json/PKG_NAME
        
        
        ## 2.1.0
        
        - Fix proxy usage. A bug in the usage of requests on our XMLRPC client
          caused this to break. You can now set `*_proxy` environment variables
          and get them picked up properly. `Fixes #59`.
        - Add a dict returned from mirror.synchronize() to show deleted
          and added files from the last run
        - Fix sorting of releases to use filename and not url
        - Tweak atomic file writes in utils.rewrite() to prefix the temporary
          file with the 'hidden' filename of the destination adding more
          support for hashed POSIX filesystems like GlusterFS. - Thanks **@cooperlees**
        
        
        # 2.0.0 (2017-04-05)
        
        - Move to Python 3. - Thanks **@cooperlees** !
        
          Official support starts with Python 3.5 but might get away with using an
          earlier version of Python 3 (maybe 3.3 or so). However, we plan to start
          using Python 3.5 features (like *asyncio*) in the near future, so please
          be advised that running with an older version of Python 3 is not
          a supported option for the long term.
        
        - General update of our dependencies to pave the road for Python 3 support.
        
        - Remove residual references to the old "statistics" script that isn't in
          use any longer.
        
        - Fix return code -- we accidentally returned 1 on successful runs
          as debugging code was mixed in the main call. `Fixes #67`.
        
        - Make the package-specific simple pages human-readable again. `Fixes #71`.
        
        
        ## 1.11 (2016-05-18)
        
        - Add option to dir-hash index files. See
          https://bitbucket.org/pypa/bandersnatch/pull-requests/22/add-option-to-dir-hash-index-files for a lot more information. Thanks
          @iwienand!
        
        - Fix an edge case: IO errors while marking off packages as "done"
          could result in crashing workers that would result in bandersnatch
          getting stuck. Thanks **@wjjt**!
        
        
        ## 1.10.0.1 (2016-05-11)
        
        - Brownbag release for re-upload. My train's Wifi broke while uploading
          ending up with a partial file on PyPI. Can your train service do better
          than mine?
        
        
        1.10 (2016-05-11)
        -----------------
        
        This is release is massively supported by **@dstufft** getting bandersnatch
        back in sync with current packaging ecosystem changes. All clap your hands
        now, please.
        
        - Refactor the generation update code to avoid weird update paths
          due to, well, my personal kink: 'over complication'.
        
        - Generate the simple index ourselves instead of copying it from PyPI.
        
        - Support files hosted on a separate domain.
        
        - Implement PEP 503 normalization rules while also providing support
          for legacy and very legacy clients.
        
        
        ## 1.9 (2016-04-21)
        
        - Fix a long standing, misunderstood bug: a non-deleting mirror would
          delete packages if they were fully removed from PyPI. `Fixes #61`
        
        
        ## 1.8 (2015-03-16)
        
        - Don't require a X-PyPI-Last-Serial header on file downloads.
          (Thanks to **@dstufft**.)
        
        - Increase our generation to help mirrors recover potential
          setuptools corruption after some data bug on PyPI.
        
        
        ## 1.7 (2014-12-14)
        
        - Fixes #54 by reordering the simple index page and file fetching
          parts. Thanks **@dstufft** for the inspiration.
        
        - Stop syncing serversig files and even start removing them.
        
        
        ## 1.6.1 (2014-09-24)
        
        - Create a new generation to enforce a full sync when upgrading.
          This is required to get the canonical names for all packages.
        
        ## 1.6 (2014-09-24)
        
        - Implement canonical package directory names to support an upcoming PIP
          release and other tools. (Thanks to **@dstufft**)
        
        - Fix a race condition where workers could get stuck indefinitely waiting for
          another item in a depleted queue. (Thanks to **@hongqn**)
        
        ## 1.5 (2014-07-21)
        
        - Delete broken tests that I forgot to remove.
        
        - Reduce the officially sanctioned maximum number of connections.
        
        ## 1.4 (2014-04-15)
        
        - Move towards replacing the XMLRPC API with JSON to make our requests
          cacheable. Also reduces the amount of requests needed dramatically.
        
        - Remove apache stats script as this information is no longer being used anyway.
        
        ## 1.3 (2014-02-16)
        
        - Move to xmlrpc2 to get SSL verification on XML-RPC calls, too. (`Fixes #40` and
          big thanks to **@ewdurbin**)
        
        ## 1.2 (2014-01-08)
        
        - Potential performance improvement: use requests' session object to allow HTTP
          pipelining. Thanks to Wouter Bolsterlee for the recommendation in `Fixes #39`.
        
        
        ## 1.1 (2013-11-26)
        
        - Made code Python 2.6 compatible. Thanks to **@ewdurbin** for the pull request.
        
        
        ## 1.0.5 (2013-07-25)
        
        - Refactor lock acquisition to avoid shadowing exceptions when creating the
          lockfile vs. acquiring the lock.
        
        - Move from distribute back to setuptools.
        
        
        ## 1.0.4 (2013-07-10)
        
        - Slight brownbag release: the requirements.txt accidentally included a
          development version of py.test due to my usage of mr.developer.
        
        ## 1.0.3 (2013-07-08)
        
        - Fix brownbag release with broken 'stable' tag and missing requirements.txt
          update.
        
        
        ## 1.0.2 (2013-07-08)
        
        - Generate the index simple page ourselves: its not signed anyway and helps
          PyPI caching more aggressively.
        
        - Add a py.test plugin to actually show a green bar. Hopefully will be
          integrated into py.test in the near future.
        
        - Fix dealing with inconsistent todo files: empty files or with an incorrect
          header will just be deleted and processing resumes at the last known good
          state.
        
        - Mark up requirement of Python 2.7 `Fixes #19`
        
        - Fix dealing with new CDN cache issues. Thanks to **@dstufft** for making PyPI
          support mirrors again.
        
        - Improve test coverage.
        
        ## 1.0.1 (2013-04-18)
        
        - Fix packaging: include default config file. (Thanks to **Jannis Leidel**)
        
        
        # 1.0 (2013-04-09)
        
        - Update pip install documentation to use the a URL for referring to the
          requirements.txt directly.
        
        - Adjust buildout and jenkins job to stop fighting over the distribute version
          to install.
        
        ## 1.0rc6 (2013-04-09)
        
        - Hopefully fixed updating the stable tag when releasing.
        
        
        ## 1.0rc5 (2013-04-09)
        
        - Experiment with zest.releaser integration to automatically generate
          requirements.txt during release process.
        
        
        ## 1.0rc4 (2013-04-09)
        -------------------
        
        - Experiment with zest.releaser integration to automatically generate
          requirements.txt during release process.
        
        
        ## 1.0rc3 (2013-04-09)
        
        - Experiment with zest.releaser integration to automatically generate
          requirements.txt during release process.
        
        
        ## 1.0rc2 (2013-04-09)
        
        - Experiment with zest.releaser integration to automatically generate
          requirements.txt during release process.
        
        
        ## 1.0rc1 (2013-04-09)
        
        - Initial release. Massive rewrite of pep381client.
        
Platform: UNKNOWN
Classifier: Programming Language :: Python :: 3 :: Only
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Description-Content-Type: text/markdown
