Diagnose Earth Engine performance with the profiler
This blog post discusses provided profiler performance numbers resulting from computations invoked by the Sentinel2-capital-greenness.js script, the basis for the 43 European capital greenness evaluation.
The execution of the whole script took about 85 seconds, or 1.42 minutes. In that time were 473 profiles viewed, 215 from API calls and 258 from the display of tiles currently visible in the map. In this particular exercise was the map zoomed in on Berlin, Germany.
The profiler recorded 57 operations and provided an output table with execution numbers
where each row corresponds to an algorithm, computation, asset load or overhead operation.
In total were 57 operations profiled.
For example, out of the 57 operations belong seven to the category Loading assets, in which the COPERNICUS/Sentinel-2 data are instantiated. Eleven times were methods called that are attached to ee.Image objects, five time for ee.ImageCollection objects. The category Other (n = 14), is an accumulation of elements that have appeared only once in their form and cannot be assigned to any other category.
Peak memory & CPU times
The numbers of the profiler indicate that the highest peak memory (50 megabytes) was needed to load the assets of the COPERNICUS/Sentinel-2 data, which took less than 10 seconds.
The second-highest peak memory (4.7 megabytes) was used on plumbing. The plumbing operation was 57.334 times invoked, thats about 48% of the total counts of operations (119.221 total counts). See the profiler operation table for details.
Definition: What does Plumbing mean? - Plumbing is a term used to describe the technology and connections between systems in a cloud computing model. It includes the systems, storage, network and the interconnection components that form the cloud environment. The quality of plumbing decides the efficiency of the applications and services delivered. - www.techopedia.com
Profiler operations table
|1||9.775||50M||1553||Loading assets: COPERNICUS/S2|
|2||7.333||1.2M||258||Encoding pixels to image|
|4||1.086||148k||12||Algorithm ImageCollection.reduce with reducer Reducer.median|
|8||0.334||229k||233||Algorithm (user-defined function)|
|10||0.202||1.0M||5||Reprojection precalculation between EPSG:32632 and SR-ORG:6627|
|14||0.193||1.0M||5||Reprojection precalculation between EPSG:32633 and SR-ORG:6627|
|15||0.074||576k||60||Loading assets: COPERNICUS/S2/20160827T101022_20160827T153533_T32UQD|
|16||0.073||1.1M||60||Loading assets: COPERNICUS/S2/20150704T101337_20160809T015434_T32UQD|
|17||0.072||958k||62||Loading assets: COPERNICUS/S2/20160827T101022_20160827T153533_T33UUU|
|18||0.071||878k||62||Loading assets: COPERNICUS/S2/20170830T102019_20170830T102531_T33UUU|
|19||0.068||1.0M||62||Loading assets: COPERNICUS/S2/20150704T101337_20160809T015434_T33UUU|
|23||0.008||1.4k||12||Reprojecting geometry to SR-ORG:6627|
|24||0.007||128||60||Reprojecting pixels from EPSG:32633 to SR-ORG:6627|
|27||0.005||616||43||Algorithm Collection.reduceColumns with reducer Reducer.toList|
|28||0.005||72k||24||Algorithm Image.clip computing pixels|
|29||0.004||128||40||Reprojecting pixels from EPSG:32632 to SR-ORG:6627|
|30||0.003||464||828||Algorithm Image.visualize computing pixels|
|31||0.002||1.0k||50||Algorithm Image.normalizedDifference computing pixels|
|32||-||321k||153||no description available|
|33||-||321k||12||Algorithm ImageCollection.reduce computing pixels|
|34||-||86k||393||Algorithm List.(construct from elements)|
|40||-||6.3k||792||Loading assets: COPERNICUS|
|54||-||2.3k||5590||Algorithm Dictionary.(construct from elements)|
|57||-||64||100||Algorithm Image.load computing pixels|
Load and filter Sentinel-2 ImageCollection
As an illustrative example for the profiler, let’s look at the following four(!) code lines where the Sentinel-2 data get loaded and a filtered ImageCollection is created.
The code creates one ImageCollection of all existing Sentinel-2 images (line 1), filters out images taken before the 166th and after the 258th day in the year (line 1). A second filter operates spatially, where all items in the collection which do not intersect the created bounding-box of Europe, will be excluded from the collection (line 2). In the next line of code are the red (B4) and near infrared (B8) bands selected and new band names given (R, N) (line 3). In the last line of code is the whole collection sorted by the image metadata property CLOUD_COVERAGE_ASSESSMENT (line 4).
Graph for Loading assets & filter ImageCollection code
You can highlight nearest nodes and edges by clicking on a node. Just click everywhere except on nodes to reset network.
If you have any questions, suggestions or spotted a mistake, please use the comment function at the bottom of this page. This blog post Diagnose Earth Engine performance with the profiler is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.