Google Earth Engine provides in the web-based JavaScript code editor a performance profiler which displays information about the resources consumed during script computation.

This blog post discusses provided profiler performance numbers resulting from computations invoked by the Sentinel2-capital-greenness.js script, the basis for the 43 European capital greenness evaluation.

Performance

The execution of the whole script took about 85 seconds, or 1.42 minutes. In that time were 473 profiles viewed, 215 from API calls and 258 from the display of tiles currently visible in the map. In this particular exercise was the map zoomed in on Berlin, Germany.

The profiler recorded 57 operations and provided an output table with execution numbers

In total were 57 operations profiled.

where each row corresponds to an algorithm, computation, asset load or overhead operation.

For example, out of the 57 operations belong seven to the category Loading assets, in which the COPERNICUS/Sentinel-2 data are instantiated. Eleven times were methods called that are attached to ee.Image objects, five time for ee.ImageCollection objects. The category Other (n = 14), is an accumulation of elements that have appeared only once in their form and cannot be assigned to any other category.

Aggregation summary of algorithm, computation, asset load and overhead operations. Aggregation summary of algorithm, computation, asset load and overhead operations.

Peak memory & CPU times

The numbers of the profiler indicate that the highest peak memory (50 megabytes) was needed to load the assets of the COPERNICUS/Sentinel-2 data, which took less than 10 seconds.

Maximum memory (50MB) was used loading assets of COPERNICUS/S2 data. Maximum memory (50MB) was used loading assets of COPERNICUS/S2 data.

The second-highest peak memory (4.7 megabytes) was used on plumbing. The plumbing operation was 57.334 times invoked, thats about 48% of the total counts of operations (119.221 total counts). See the profiler operation table for details.

Definition: What does Plumbing mean? - Plumbing is a term used to describe the technology and connections between systems in a cloud computing model. It includes the systems, storage, network and the interconnection components that form the cloud environment. The quality of plumbing decides the efficiency of the applications and services delivered. - www.techopedia.com

Profiler operations table

ID Compute Peak.Mem Count Description
1 9.775 50M 1553 Loading assets: COPERNICUS/S2
2 7.333 1.2M 258 Encoding pixels to image
3 5.143 4.7M 57334 (plumbing)
4 1.086 148k 12 Algorithm ImageCollection.reduce with reducer Reducer.median
5 0.431 45k 3554 Algorithm Image.select
6 0.337 17k 1763 Algorithm Image.visualize
7 0.334 7.0k 903 Algorithm Image.clip
8 0.334 229k 233 Algorithm (user-defined function)
9 0.211 34k 1777 Algorithm Image.float
10 0.202 1.0M 5 Reprojection precalculation between EPSG:32632 and SR-ORG:6627
11 0.202 33k 1777 Algorithm Image.addBands
12 0.198 23k 1777 Algorithm Image.rename
13 0.196 19k 1777 Algorithm Image.normalizedDifference
14 0.193 1.0M 5 Reprojection precalculation between EPSG:32633 and SR-ORG:6627
15 0.074 576k 60 Loading assets: COPERNICUS/S2/20160827T101022_20160827T153533_T32UQD
16 0.073 1.1M 60 Loading assets: COPERNICUS/S2/20150704T101337_20160809T015434_T32UQD
17 0.072 958k 62 Loading assets: COPERNICUS/S2/20160827T101022_20160827T153533_T33UUU
18 0.071 878k 62 Loading assets: COPERNICUS/S2/20170830T102019_20170830T102531_T33UUU
19 0.068 1.0M 62 Loading assets: COPERNICUS/S2/20150704T101337_20160809T015434_T33UUU
20 0.060 6.1k 178 Listing collection
21 0.051 473k 921 Algorithm ImageCollection.reduce
22 0.022 4.2k 387 Algorithm Geometry.buffer
23 0.008 1.4k 12 Reprojecting geometry to SR-ORG:6627
24 0.007 128 60 Reprojecting pixels from EPSG:32633 to SR-ORG:6627
25 0.006 4.0k 516 Algorithm Element.geometry
26 0.006 12k 344 Algorithm Collection.geometry
27 0.005 616 43 Algorithm Collection.reduceColumns with reducer Reducer.toList
28 0.005 72k 24 Algorithm Image.clip computing pixels
29 0.004 128 40 Reprojecting pixels from EPSG:32632 to SR-ORG:6627
30 0.003 464 828 Algorithm Image.visualize computing pixels
31 0.002 1.0k 50 Algorithm Image.normalizedDifference computing pixels
32 - 321k 153 no description available
33 - 321k 12 Algorithm ImageCollection.reduce computing pixels
34 - 86k 393 Algorithm List.(construct from elements)
35 - 67k 387 Algorithm Collection
36 - 21k 3150 Algorithm Collection.map
37 - 11k 921 Algorithm ImageCollection.load
38 - 10k 17415 Algorithm Feature
39 - 7.4k 129 Algorithm AggregateFeatureCollection.array
40 - 6.3k 792 Loading assets: COPERNICUS
41 - 5.6k 86 Algorithm Collection.reduceColumns
42 - 5.4k 792 Algorithm TypedImageCollection.Constructor
43 - 5.3k 3150 Algorithm Collection.filter
44 - 4.7k 2763 Algorithm Collection.limit
45 - 4.5k 1842 Algorithm Filter.intersects
46 - 4.4k 792 Algorithm PixelType
47 - 4.3k 387 Algorithm Element.copyProperties
48 - 4.3k 792 Algorithm Filter.calendarRange
49 - 4.2k 387 Algorithm Filter.eq
50 - 4.1k 129 Algorithm Filter.dayOfYear
51 - 4.1k 792 Algorithm ErrorMargin
52 - 4.0k 258 Algorithm String
53 - 3.5k 215 Algorithm List.get
54 - 2.3k 5590 Algorithm Dictionary.(construct from elements)
55 - 1.1k 307 Algorithm Reducer.median
56 - 256 1050 Expression evaluation
57 - 64 100 Algorithm Image.load computing pixels

Load and filter Sentinel-2 ImageCollection

As an illustrative example for the profiler, let’s look at the following four(!) code lines where the Sentinel-2 data get loaded and a filtered ImageCollection is created.

Loading assets & filter ImageCollection
1
2
3
4
var s2 = ee.ImageCollection('COPERNICUS/S2').filter(ee.Filter.dayOfYear(166,258)) 
    .filterBounds(Europe_bbox.geometry())
    .select(['B4', 'B8', ], ['R','N'])
    .sort('CLOUD_COVERAGE_ASSESSMENT');

The code creates one ImageCollection of all existing Sentinel-2 images (line 1), filters out images taken before the 166th and after the 258th day in the year (line 1). A second filter operates spatially, where all items in the collection which do not intersect the created bounding-box of Europe, will be excluded from the collection (line 2). In the next line of code are the red (B4) and near infrared (B8) bands selected and new band names given (R, N) (line 3). In the last line of code is the whole collection sorted by the image metadata property CLOUD_COVERAGE_ASSESSMENT (line 4).

Graph for Loading assets & filter ImageCollection code

You can highlight nearest nodes and edges by clicking on a node. Just click everywhere except on nodes to reset network.


Bottom line

If you have any questions, suggestions or spotted a mistake, please use the comment function at the bottom of this page. This blog post Diagnose Earth Engine performance with the profiler is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Previous blog posts are available within the blog archive. Feel free to connect or follow me on twitter - @gartn001.