Solar Design and Satellite Image Segmentation

1. Satellite Image Segmentation
for
Photovoltaic Potential Estimation
Python based Google Cloud Platform project
Johannes Oos
oosjoh@gmail.com

2. 1.Purpose and Input Data
2.Computer Vision/ Models
3.Auto-ML
4.Summary

3. 1.Purpose and Input Data
2.Computer Vision/ Models
3.Auto-ML
4.Summary

4. Purpose
Enter address
Estimation of
system key
parameters:
Size, energy yield

5. Address Size, tilt,
azimuth
climate
data
Solar
Simulation
Size,
Energy
Data Flow
Geo-location
lat / lng

6. Datasource – Images from googlemaps
Geocoding API
• To translate an address to geo-coordinates
• Clean and Python friendly
• pip install -U googlemaps
Maps Static API
• To get the satelite image via http request
• URL parsing with urllib

7. Geocoding API

8. Maps Static API - request

9. Maps Static API - image

10. 1.Purpose and Input Data
2.Computer Vision/ Models
3.Auto-ML
4.Summary

11. Image Analysis

12. Image Analysis
Classification Detection Segmentation
90%
http://cocodataset.org/#explore

13. Model Family Task Input Output Technology Model
1 Segmentation Create Roof Segments,
Determine size of
roofs
Satellite Image Roof Box,
Segments
Keras
TensorFlow
Mask R-CNN
2 Classification Decide if roof is flat or
tilted
Roof Box Image Label AutoML GCP
3 Classification Get Tilt angle Roof Box Image of
tilted roof
Label AutoML GCP
4 Classification Get Azimuth angle Roof Box Image of
tilted roof
Label AutoML GCP
Overview – Used Models

14. Model Family Task Input Output Technology Model
1 Segmentation Create Roof Segments,
Determine size of
roofs
Satellite Image Roof Box,
Segments
Keras
TensorFlow
Mask R-CNN
2 Classification Decide if roof is flat or
tilted
Roof Box Image Label AutoML GCP
3 Classification Get Tilt angle Roof Box Image of
tilted roof
Label AutoML GCP
4 Classification Get Azimuth angle Roof Box Image of
tilted roof
Label AutoML GCP
Overview – Used Models

15. Mask R-CNN
• https://arxiv.org/abs/1703.06870
• https://github.com/matterport/Mask_RCNN
• A well documented, easy to use implementation for image
segmentation tasks
• Implemented with Python, Tensorflow, Keras

16. Mask R-CNN
Blue Bounding Box:
Starting top Left [ 79, 15, 164, 68]
Pixels in the Blue Mask:
== > 2842 px
@Zoom Level 19:
Distance è 0.2986 m / pixel
Area è 11 pixel / sqm
Blue roof = 260 sqm
The content of the Bounding Box
Is used for classification

17. Model Family Task Input Output Technology Model
1 Segmentation Create Roof Segments,
Determine size of
roofs
Satellite Image Roof Box,
Segments
Keras
TensorFlow
Mask R-CNN
2 Classification Decide if roof is flat or
tilted
Roof Box Image Label AutoML GCP
3 Classification Get Tilt angle Roof Box Image of
tilted roof
Label AutoML GCP
4 Classification Get Azimuth angle Roof Box Image of
tilted roof
Label AutoML GCP
Overview – Used Models

18. Satellite images
: “flat”
: “tilted”
Label data set
GCP AutoML
Customized
Classifier
91% Tilted
Pre-trained Classifier
Retraining a Classifier

19. Choice of Classifier

20. 1.Purpose and Input Data
2.Computer Vision/ Models
3.Auto-ML
4.Summary

21. ML APIs MLAutoML
Train high-quality custom machine learning models
with minimum effort and machine learning expertise
AutoML on GCP
Data
Scientist
App
developer

22. Vision
Natural
Language
Translate
AutoML on GCP

23. AutoML Admin Prerequisites
• Set Up GCP Account
• Enable Billing
• Create GCP Project
• Authorization
• Pip install API
• Enable API in project

24. Admin Prerequisites – Set Credentials
DefaultCredentialsError: Could not automatically determine
credentials. Please set GOOGLE_APPLICATION_CREDENTIALS

25. Admin Prerequisites – Enable API
PermissionDenied: 403 […] API has not been used in project
[…] before or it is disabled

26. Train – Uploading and Labelling the data

27. Train and Evaluate the models

28. Train – Keeping track of the models

29. AutoML - Using the ready model (1/2)
Image
Type: google.cloud.automl_v1beta1.
types.PredictResponse
payload { classification {
score: 0.7124295830726624 }
display_name: "roof_20"}
res.payload[0].display_name
Personalized Classifier
Hosted on GCP
Response
Object
Timing: 1 - 3 Seconds

30. AutoML - Using the ready model (2/2)

31. 1.Purpose and Input Data
2.Computer Vision/ Models
3.Auto-ML
4.Summary

32. Bringing it all together
Size: 260 sqm
”Address”
”lat / lng”
Irradiation Profile
30 kWp @
950 kWh/kWp/year Simulation
Type: Flat

33. Thank you!

34. Using AutoML
1. Very comfortable data and model management
2. Python friendly usage
3. Still in Beta mode
4. Good for Classification
5. Can’t be optimized for speed