# 15-分布式爬虫的部署

在前一章我们成功实现了分布式爬虫，但是在这个过程中我们发现有很多不方便的地方。

在将 Scrapy 项目放到各台主机运行时，你可能采用的是文件上传或者 Git 同步的方式，但这样需要各台主机都进行操作，如果有 100 台、1000 台主机，那工作量可想而知。

本章我们就来了解一下，分布式爬虫部署方面可以采取的一些措施，以方便地实现批量部署和管理。


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://python3webspider.cuiqingcai.com/15-fen-bu-shi-pa-chong-de-bu-shu.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.