nsw_lidar
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
nsw_lidar [2020/08/31 23:22] – [Resources] allchin09 | nsw_lidar [2021/02/09 23:02] – bushwalking | ||
---|---|---|---|
Line 10: | Line 10: | ||
* [[https:// | * [[https:// | ||
* [[http:// | * [[http:// | ||
+ | |||
+ | ====== Managing DEMs ====== | ||
+ | |||
+ | ===== Downloading data ===== | ||
+ | |||
+ | While data can be downloaded in an ad hoc manner, if you are regularly processing DEMs, it is better to have the DEM tiles already downloaded. Download tiles by 1:100k map area, which is 0.5 x 0.5 degree squares. Each 1:100k map area ranges from around 2GB to 6GB of data, depending on the number of 2m DEMs vs 1m DEMs, and other factors. | ||
+ | |||
+ | The Sydney basin and Blue Mountains is around 50GB all up. | ||
+ | |||
+ | For example, to request data for the Katoomba 1:100k map area (-33.5, | ||
+ | {{:: | ||
+ | |||
+ | Below are the 1:100k map areas around Sydney: | ||
+ | |||
+ | |8833_gulgong|8933_merriwa|9033_muswellbrook|9133_camberwell|9233_dungog|9333_buladelah|9433_forster| | ||
+ | |8832_mudgee|8932_mt_pomany|9032_howes_valley|9132_cessnock|9232_newcastle|9332_port_stephens| | | ||
+ | |8831_bathurst|8931_wallerawang|9031_st_albans|9131_gosford|9231_lake_macquarie| | | | ||
+ | |8830_oberon|8930_katoomba|9030_penrith|9130_sydney| | | | | ||
+ | |8829_taralga|8929_burragorang|9029_wollongong|9129_port_hacking| | | | | ||
+ | |8828_goulburn|8928_moss_vale|9028_kiama| | | | | | ||
+ | |8827_braidwood|8927_ulladulla|9027_jervis_bay| | | | | | ||
+ | |||
+ | ===== Pre-processing data ===== | ||
+ | The following may be useful for Windows users. | ||
+ | |||
+ | Below is a Windows Powershell script that will | ||
+ | * move any old DEMs and DEMs from a different zone (you can't mix zones in a virtual raster) to an archive sub-folder | ||
+ | * extract the raw data from the remaining zip files | ||
+ | * convert all of the .ASC files to GeoTIFF | ||
+ | * move the old zip files to a current sub-folder | ||
+ | * zip the current and archive sub-folders to temp.zip | ||
+ | * create a virtual raster (.vrt) file of all of the GeoTiffs | ||
+ | |||
+ | You will need to replace the Environment variables with your own - lines starting with $Env. | ||
+ | |||
+ | Usage is: buildvrt.ps1 < | ||
+ | eg buildvrt.ps1 < | ||
+ | |||
+ | <file powershell buildvrt.ps1> | ||
+ | $zipFile=$args[0] | ||
+ | $targetFolder=$args[1] | ||
+ | |||
+ | # Unzip files from all subdirectories to new folder | ||
+ | Expand-Archive -LiteralPath $zipFile -DestinationPath $targetFolder | ||
+ | Get-ChildItem -Path " | ||
+ | Get-ChildItem -Path $targetFolder -Directory | Remove-Item -Recurse | ||
+ | |||
+ | # Create hash of zip files, by name (location, date) | ||
+ | $zipFileList = @{} | ||
+ | Get-ChildItem -Path " | ||
+ | $zipFileList.add($_, | ||
+ | $_ -match ' | ||
+ | $zipFileList[$_][' | ||
+ | $zipFileList[$_][' | ||
+ | $_ -match ' | ||
+ | $zipFileList[$_][' | ||
+ | } | ||
+ | # $zipFileList | ConvertTo-Json | ||
+ | |||
+ | # Create hash of location (date, name) | ||
+ | $locationList = @{} | ||
+ | $zoneCount = @{} | ||
+ | $zipFileList.keys | ForEach-Object { | ||
+ | if($locationList[$zipFileList[$_][' | ||
+ | $t = $locationList[$zipFileList[$_][' | ||
+ | $t.add($zipFileList[$_][' | ||
+ | } else { | ||
+ | $t = @{} | ||
+ | $t.add($zipFileList[$_][' | ||
+ | $locationList.add($zipFileList[$_][' | ||
+ | } | ||
+ | if($zoneCount[$zipFileList[$_][' | ||
+ | $zoneCount[$zipFileList[$_][' | ||
+ | } else { | ||
+ | $zoneCount[$zipFileList[$_][' | ||
+ | } | ||
+ | } | ||
+ | # $locationList | ConvertTo-Json | ||
+ | # $zoneCount | ConvertTo-Json | ||
+ | |||
+ | # Create archive folder | ||
+ | $archiveFolder = " | ||
+ | If(!(test-path $archiveFolder)) | ||
+ | { | ||
+ | New-Item -ItemType Directory -Force -Path $archiveFolder | ||
+ | } | ||
+ | |||
+ | # Sort each location by date desc, and move old files to /archive | ||
+ | $locationList.keys | ForEach-Object { | ||
+ | $i=0 | ||
+ | $locationList[$_].GetEnumerator() | sort key -des | ForEach-Object { | ||
+ | if ($i -eq 0) { | ||
+ | $i++ | ||
+ | return} | ||
+ | else { | ||
+ | #$_ | ConvertTo-Json | ||
+ | $s = $_.Value | ||
+ | Move-Item -Path " | ||
+ | } | ||
+ | } | ||
+ | } | ||
+ | |||
+ | # You can't build a VRT with files from a different projection, so | ||
+ | # delete files from outside main zone | ||
+ | # This could probably be altered to include a step to reproject those files | ||
+ | $mainZone = '' | ||
+ | $zoneCount.GetEnumerator() | sort value -des | select -first 1 | ForEach-Object { | ||
+ | $mainZone = $_.Name | ||
+ | } | ||
+ | |||
+ | $zipFileList.keys | ForEach-Object { | ||
+ | if($zipFileList[$_][' | ||
+ | Remove-Item -Path " | ||
+ | } | ||
+ | } | ||
+ | |||
+ | Get-ChildItem -Path " | ||
+ | $Env:Path += "; | ||
+ | $Env: | ||
+ | $Env: | ||
+ | $Env: | ||
+ | |||
+ | Get-ChildItem -Path " | ||
+ | $srcFile = $_.FullName | ||
+ | $destFile = $_.FullName -replace ' | ||
+ | &" | ||
+ | } | ||
+ | |||
+ | Get-ChildItem -Path " | ||
+ | Get-ChildItem -Path " | ||
+ | Get-ChildItem -Path " | ||
+ | Get-ChildItem -Path " | ||
+ | |||
+ | # Create current folder | ||
+ | $currentFolder = " | ||
+ | If(!(test-path $currentFolder)) | ||
+ | { | ||
+ | New-Item -ItemType Directory -Force -Path $currentFolder | ||
+ | } | ||
+ | |||
+ | # Move zip files to current folder | ||
+ | Move-Item -Path " | ||
+ | |||
+ | # Zip /archive & /current to new zip folder | ||
+ | Compress-Archive -Path " | ||
+ | |||
+ | # Create 2m vrt | ||
+ | New-Item " | ||
+ | Get-ChildItem -Path " | ||
+ | &" | ||
+ | </ | ||
+ | |||
+ | It is possible to then build a larger virtual raster from the individual 1:100k virtual rasters. | ||
+ | |||
+ | ===== Loading data ===== | ||
+ | |||
+ | Loading up a large virtual raster into QGIS can be very slow, as can manipulating it. However, you can quickly load up a smaller section of the map using the following steps: | ||
+ | * create a polygon using https:// | ||
+ | * download the polygon using the **Export drawn data to GeoJSON** function | ||
+ | * load up the Python console (// | ||
+ | * run the following command (replace file locations with your own) | ||
+ | |||
+ | < | ||
+ | resultClip = processing.runAndLoadResults(" | ||
+ | </ | ||
====== Topographic maps ====== | ====== Topographic maps ====== | ||
Line 228: | Line 393: | ||
[[http:// | [[http:// | ||
+ | |||
+ | [[https:// | ||
+ | |||
+ | [[https:// | ||
==== Method ==== | ==== Method ==== | ||
The below snip of Breakfast Creek makes use of TPI calculated from a LIDAR derived DEM. Only positive values for TPI as displayed, which indicate cliff-like features. It is then combined with contours and aerial imagery to convey the terrain of the area. | The below snip of Breakfast Creek makes use of TPI calculated from a LIDAR derived DEM. Only positive values for TPI as displayed, which indicate cliff-like features. It is then combined with contours and aerial imagery to convey the terrain of the area. |
nsw_lidar.txt · Last modified: 2024/04/05 19:00 by bushwalking