When generating caches for raster tiles generated using map image layers and imagery layers, there are a number of tools and processes you need to be familiar with.
Note:
The term map service can be used interchangeably with the term map image layer, and the term image service can be used interchangeably with the term imagery layers.
Caching toolset
Cache management tools for server services such as cached map layers, image layers, and tile layers are located in the Server Tools > caching toolset in ArcGIS Pro. See An overview of the Caching toolset for more information.
Note:
All caching tools require publisher or administrator credentials to use.
Server resources for caching
Generating and managing caches in ArcGIS Server is supported by a preconfigured caching controller and caching tool geoprocessing services. These system services are readily available when you create your ArcGIS Server site. The number of instances you allow for the CachingTools service determines how much power your machine can dedicate to caching jobs, while the number of Caching controller service instances in your System folder determines how many jobs can run at the same time.
Increasing the number of instances of a map or image service will not affect how fast tiles are created. The map or image service instances are used to fulfill the query and identify operations on the service and are not used for rendering the cached content. You can decrease the load on your server resources by taking advantage of shared instances for cached map services by configuring pooling of your map and image services. See Configure service instance settings for information.
Choose the number of instances for the CachingTools service
You can use ArcGIS Server Manager at any time to adjust the maximum number of instances of the CachingTools geoprocessing service that you want to make available for caching jobs. The minimum and maximum values apply to each individual GIS server, and as such, if the maximum is set to a value of 3 and you have four GIS servers running the CachingTools service, you could have up to 12 instances of CachingTools running.
This behavior allows you to add and remove GIS servers from your site to increase or decrease the number of resources dedicated to caching. You can add a GIS server even when a caching job is running, and it will be detected and assigned tiles to be created.
It is important that you choose an appropriate amount of instances of the CachingTools service. If you allow too many instances, your machine can become overwhelmed and inefficient. However, if you allow too few, your machine can go underutilized. Finding the best number for your circumstances can be a process of trial and error. However, it is recommended that you start by allowing a maximum of n, where n is the number of CPU cores on a single machine in your cluster.
Choose the number of instances for a caching job
Tools such as Manage Map Server Cache Tiles allow you to choose how many instances of CachingTools will work on the job. You can choose to divide the available instances of CachingTools among several running jobs. A job might not use its maximum number of instances of CachingTools if those instances are being used by other jobs. If a caching job is using all the CachingTools instances, other requested jobs are queued until the first job finishes.
For example, you want to create a cache and you have four GIS servers in a site. You've configured each server to allow a maximum of five instances of CachingTools. The maximum number of instances you can dedicate toward any caching job is 20. If you want to run two simultaneous caching jobs on this site and maintain an evenly distributed load, you can dedicate 10 instances toward each job.
Allow for elasticity
It may be that you configured your site in a cloud environment that can automatically add GIS servers in response to demand. You may not want to be limited by a fixed maximum number of instances that can work on the job. In this case, you can enter a value of -1 to indicate that there is no limit to the number of instances that can work on the job. All the available instances of CachingTools will be used for the job, no matter how many GIS servers are added to your site.
Set the number of jobs that can run simultaneously
If too many publishers start requesting cache to be built at the same time, the server can get overwhelmed, even if you only choose to dedicate a small number of instances toward each job. The CachingControllers service (in the System folder) determines how many jobs can run at the same time.
The default maximum number of instances for the CachingControllers service is 3, meaning only three caching jobs can run at once. If the server receives a request for a fourth caching job, it will be queued until one of the other jobs has finished. If you want to allow four jobs to run at once, you can set the maximum number of instances of CachingControllers to 4.
Automate cache creation and updates with geoprocessing
If you're working with a basemap that is unlikely to change, you'll probably create its cache only once. You can run ArcGIS geoprocessing tools to create the cache and add tiles to it. This is also an appropriate way to update the cache if the data does not change very often.
If the data changes often, you still may be able to use caching. The caching tools are designed to allow you to script cache updates and target specific parts of the cache for updates. For example, you can do the following:
- Update the cache only for scale levels that have changed.
- Update only areas of the cache that fall within the boundaries of a feature class.
- Automate cache maintenance with utilities such as Windows Task Scheduler.
The way to automate the caching process is to write scripts that use tools from the Caching toolset in the Server Tools toolbox.
To create a map or image service cache, you will typically start with the Create Map Server Cache tool, which initializes the cache. Then you run the Manage Map Server Cache Tiles to add tiles to the cache.
There are many tools that work with exporting, importing, and deleting caches. For a summary of the available tools, see An overview of the Caching toolset.
Use the tools in scripts
Although you can manually open the caching tools from the Catalog pane or Search window, it's more efficient to create a model in ModelBuilder or write a geoprocessing script that contains one or more tools that you want to run. You can then schedule scripts to run automatically.
The geoprocessing tool reference topic contains a full Python code sample showing how to use the tool in a script. For example, see the help for Manage Map Server Cache Tiles.
If you're new to Python, these resources can help you learn more:
- What is Python?—The ArcGIS help contains an introduction to Python scripting, starting with this topic.
- Exporting a model—If you've created models in ModelBuilder, you can export them to Python to see what the corresponding script looks like.
Schedule a script to run on a regular basis
You can save a script as a Python script file (.py). You can double-click the script in Microsoft File Explorer to run it. This is useful for testing, but in most cases, you'll want to schedule the script to run automatically on a regular basis.
Your operating system contains utilities that can help you schedule tasks such as running a script. In Windows, you can use Task Scheduler or the schtasks command. You need to provide the location of the script file, how often you want it to run, and the name and password that the task will run as. For more information, see Scheduling a Python script to run at prescribed times.
If the cache process crashes
Cache generation work submitted by clients such as ArcGIS Pro are submitted to the caching controller geoprocessing service that divides jobs into bundle extents and submits them to the caching tools (worker) geoprocessing service. The caching tools geoprocessing service then renders the map or image service and generates the tiles for it. Sometimes this process can crash before marking the job as either succeeded or failed. In these instances, the caching controller may continue to wait for the process to complete.
The maxTimeForWorkerJob property in the caching controller geoprocessing service manages the maximum time the caching controller geoprocessing service will wait before marking the job as cancelled. An administrator can update the value for maxTimeForWorkerJob to be greater than the average time for a bundle extent for a given level.
For example, if creating a cache for a bundle extent at level 19 takes an average of 10 minutes, it is recommended that you set the value for maxTimeForWorkerJob to 20 minutes. The default value for maxTimeForWorkerJob is 300 minutes.