Trying to open a mesh layer as user fails in QGIS with the following error: But when you run QGIS as admin, the same layer can be loaded. Note, this happened for a 2dm file.

If I try to open this file: https://github.com/lutraconsulting/MDAL/blob/master/tests/data/ugrid/D-Flow1.2/bw_11_zonder_riviergrid_met_1dwtg_clm.nc

It causes core dump. To reproduce it, you can try and add it twice (while the first one is open).

Fudaa-prepro is used for pre and post processing of TELEMAC2D model.

I have an "invalid string position" when importing a mesh exported from fudaa (see attached file). Qgis 3.14.1

export_fudaa.zip

WHen MDAL is built on GDAL 3.4 - the GRIB tests fail with the following message

11: C:\Users\runes\Documents\GitHub\MDAL\tests\test_gdal_grib.cpp(214): error: Expected equality of these values: 11: -0.818756103515625 11: value 11: Which is: -0.535552978515625 This is consistent across all platforms that I have tried.

I believe that this might be related to https://github.com/OSGeo/gdal/issues/4524 since - if you run gdalinfo on the test GRIB file you get different results for different versions of GDAL - note the origin and corner coordinates. I do not know enough about the GRIB driver to know what effect this change would have.

IN 3.3.3 you get Warning: Inside GRIB2Inventory, Message # 3 ERROR: Couldn't find 'GRIB' or 'TDLP' Driver: GRIB/GRIdded Binary (.grb, .grb2) Files: wind_only_u_component.grib Size is 480, 241 Coordinate System is: GEOGCRS["Coordinate System imported from GRIB file", DATUM["unnamed", ELLIPSOID["Sphere",6367470,0, LENGTHUNIT["metre",1, ID["EPSG",9001]]]], PRIMEM["Greenwich",0, ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]], CS[ellipsoidal,2], AXIS["latitude",north, ORDER[1], ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]], AXIS["longitude",east, ORDER[2], ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]]] Data axis to CRS axis mapping: 2,1 Origin = (-0.375000000000000,90.375000000000000) Pixel Size = (0.750000000000000,-0.750000000000000) Corner Coordinates: Upper Left ( -0.3750000, 90.3750000) ( 0d22'30.00"W, 90d22'30.00"N) Lower Left ( -0.3750000, -90.3750000) ( 0d22'30.00"W, 90d22'30.00"S) Upper Right ( 359.625, 90.375) (359d37'30.00"E, 90d22'30.00"N) Lower Right ( 359.625, -90.375) (359d37'30.00"E, 90d22'30.00"S) Center ( 179.6250000, 0.0000000) (179d37'30.00"E, 0d 0' 0.01"N) Band 1 Block=480x1 Type=Float64, ColorInterp=Undefined Description = 0[-] SFC (Ground or water surface) Metadata: GRIB_COMMENT=10 metre u wind component [m/s] GRIB_ELEMENT=10U GRIB_FORECAST_SECONDS=43200 sec GRIB_REF_TIME= 1538352000 sec UTC GRIB_SHORT_NAME=0-SFC GRIB_UNIT=[m/s] GRIB_VALID_TIME= 1538395200 sec UTC Band 2 Block=480x1 Type=Float64, ColorInterp=Undefined Description = 0[-] SFC (Ground or water surface) Metadata: GRIB_COMMENT=10 metre u wind component [m/s] GRIB_ELEMENT=10U GRIB_FORECAST_SECONDS=43200 sec GRIB_REF_TIME= 1538395200 sec UTC GRIB_SHORT_NAME=0-SFC GRIB_UNIT=[m/s] GRIB_VALID_TIME= 1538438400 sec UTC In 3.4 you get Warning: Inside GRIB2Inventory, Message # 3 ERROR: Couldn't find 'GRIB' or 'TDLP' Driver: GRIB/GRIdded Binary (.grb, .grb2) Files: wind_only_u_component.grib Size is 480, 241 Coordinate System is: GEOGCRS["Coordinate System imported from GRIB file", DATUM["unnamed", ELLIPSOID["Sphere",6367470,0, LENGTHUNIT["metre",1, ID["EPSG",9001]]]], PRIMEM["Greenwich",0, ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]], CS[ellipsoidal,2], AXIS["latitude",north, ORDER[1], ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]], AXIS["longitude",east, ORDER[2], ANGLEUNIT["degree",0.0174532925199433, ID["EPSG",9122]]]] Data axis to CRS axis mapping: 2,1 Origin = (-180.375000000000000,90.375000000000000) Pixel Size = (0.750000000000000,-0.750000000000000) Corner Coordinates: Upper Left (-180.3750000, 90.3750000) (180d22'30.00"W, 90d22'30.00"N) Lower Left (-180.3750000, -90.3750000) (180d22'30.00"W, 90d22'30.00"S) Upper Right ( 179.6250000, 90.3750000) (179d37'30.00"E, 90d22'30.00"N) Lower Right ( 179.6250000, -90.3750000) (179d37'30.00"E, 90d22'30.00"S) Center ( -0.3750000, 0.0000000) ( 0d22'30.00"W, 0d 0' 0.01"N) Band 1 Block=480x1 Type=Float64, ColorInterp=Undefined Description = 0[-] SFC (Ground or water surface) Metadata: GRIB_COMMENT=10 metre u wind component [m/s] GRIB_ELEMENT=10U GRIB_FORECAST_SECONDS=43200 GRIB_REF_TIME=1538352000 GRIB_SHORT_NAME=0-SFC GRIB_UNIT=[m/s] GRIB_VALID_TIME=1538395200 Band 2 Block=480x1 Type=Float64, ColorInterp=Undefined Description = 0[-] SFC (Ground or water surface) Metadata: GRIB_COMMENT=10 metre u wind component [m/s] GRIB_ELEMENT=10U GRIB_FORECAST_SECONDS=43200 GRIB_REF_TIME=1538395200 GRIB_SHORT_NAME=0-SFC GRIB_UNIT=[m/s] GRIB_VALID_TIME=1538438400

https://apps.ecmwf.int/datasets/data/interim-full-daily/levtype=sfc/

crashes on

void MDAL::DriverGdal::addDataToOutput( GDALRasterBandH raster_band, std::shared_ptr<MemoryDataset> tos, bool is_vector, bool is_x )
{
  assert( raster_band );

where raster_band is null pointer;

This PR is the first part of an update to the PLY driver to make to READ/WRITE and to include binary as well as ASCII format files. Currently, although it does both formats it has only been tested on ASCII formats. There will be an update when the binary formats are tested and working (which will be dependent on the metadata PR submitted separately). Submitting now to start the review.

NOTES:

1 This driver has been built on top of an existing library for PLY. This code is fully MIT licenced (include explicit language allowing the code to be merged) and I have included the copyright notice both in the MDAL LICENCE.TXT and in its own licence file. I believe this meets all legal requirements. Note that the libplyxx code has been extensively customised to the needs of MDAL, has been changed to use MDAL error logging and has been updated to include all PLY data types and lists in all positions.

2 I have build the PLY library code into the MDAL shared object to avoid problems with creating new dependencies. This may be seen as not usual - but the amount of code involved is no more than would be required to write a PLY driver from scratch and it does not bring in any unusual or new dependencies itself (except as in the next point) - so this seems reasonable to me.

3 This code DOES make extensive use of std::unordered_map and therefore has a dependency on C++ 14. This does not seem unreasonable in 2021.

4 The use of libplyxx approach means that the actual mdal_ply code is such simplified down to just wrangling the MDAL into and out of the libplyxx data formats.

I've been working with BASEMENT and up to now the visualization on Crayfish was fine. With QGIS update to v3.4 and the MDAL tool I'm struggling in visualizing BASEMENT outputs in qgis. When trying to add both ascii.dat or *.sol files as extra dataset to the 2dm mesh, the message "Could not read mesh dataset." appears. Actually, by reading these output files, the *.sol files are written in the *.dat syntax, but I am unable to upload them on MDAL as extra dataset.

I tried to create the simplest possible example for test purposes - see http://forum.aquaveo.com/topic/3426-working-example-of-a-2dm-mesh-file-with-e6t-elements-six-node-quadratic-triangles/

Unfortunately, I haven't received any responses from the SMS forum.

In QGIS, the file seems to read fine, but doesn't render:

Crayfish exports the nodes, but produces an empty (nodata) raster and empty polygon for the face (expression used to create the "wkt" virtual field - geom_to_wkt( $geometry) ) :

QGIS version info:


QGIS version
3.10.2-A Coruña
QGIS code revision
d4cd3cfe5a
Compiled against Qt
5.11.2
Running against Qt
5.11.2
Compiled against GDAL/OGR
3.0.3
Running against GDAL/OGR
3.0.3
Compiled against GEOS
3.8.0-CAPI-1.13.1
Running against GEOS
3.8.0-CAPI-1.13.1 
Compiled against SQLite
3.29.0
Running against SQLite
3.29.0
PostgreSQL Client Version
11.5
SpatiaLite Version
4.3.0
QWT Version
6.1.3
QScintilla2 Version
2.10.8
Compiled against PROJ
6.3.0
Running against PROJ
Rel. 6.3.0, January 1st, 2020
OS Version
Windows 10 (10.0)
Active python plugins
changeDataSource; 
crayfish; 
gmsh; 
latlontools; 
Mergin; 
mmqgis; 
profiletool; 
QMetaTiles; 
quick_map_services; 
shapetools; 
splitmultipart; 
SVG2ColoR; 
timemanager; 
valuetool; 
VectorMCDA; 
VoGisProfilTool; 
db_manager; 
MetaSearch; 
processing

Hi! We're using mdal/crayfish quite often, especially to show and analyse results of the 2d hydraulic model hydro-as-2d / hydro-ft-2d. ...and it would be great, if we can show and analyse the result from .h5 files in QGIS :)

Is a driver for the xmdf (.h5) format planned to get into mdal? maybe its just an extension to the already existing xmdf driver. hydro-as stores a relatively simple xmdf format with one group per dataset. Each group stores 4 tables (Mins, Maxs, Times, Values) I'm not into cpp, but I wrote a simple function in python using the h5py lib (see attached) to read/write the data. May this could help a little bit to get the driver for reading these files into mdal :)

hydras_h5_io.py.zip

test files testdata_hydro_as.zip

it would be great to see the hydro-as xmdf support in the future!! cheers

Time to go ahead with GDAL >=3.4, and test using ubuntu latest

For windows, tests are done after loading dependencies from OSGEO instead of chocolatey.

For macos, not sure GDAL >=3.4 is easily available (@PeterPetrik ?), sor for now macos still uses old version.

These changes lead to drop the test coverage (-0.6), so some works are done to make this coverage increase.

fix #391

Hi I'm an old school informatic user so I often create directory and file without any spaces but with shared space, I'm not always the decider... I recently encountered a problem with a netcdf ugrid file which is well open by QGIS/MDAL when located on my computer and where a crash occur if I open the same file located on a server. The only difference I saw is the name of the path which contains spaces on the server architecture. As other files (gpkg, shp, ...) are well opened by QGIS, I suspect this in an internal problem in MDAL but this need to be confirmed and potentially corrected Thanks in advance

We should read the GRIB/NetCDF formats natively, GDAL is not a dependency for a low-level library as MDAL.

related #416 related #391 related #273 related #237 duplicate #165 related #106 related #103

For both, binary and ascii dataset files the index (position/line) of the values correspond to the node numbers in the mesh. In most cases nodes are numbered from 1 to maximumId. But this is not mandatory. There may be reasons to remove some nodes from a mesh without renumbering, having the ability to use existing datasets. If I use ascii format this is no problem in most cases, because the validation of the dataset is made by comparision of the value behind the ND-card and the maximum ID of mesh nodes (see mdal_ascii_dat.cpp, MDAL::DriverAsciiDat::loadNewFormat/loadOldFormat):

    if ( cardType == "ND" && items.size() >= 2 )
    {
      size_t fileNodeCount = toSizeT( items[1] );
      size_t meshIdCount = maximumId( mesh ) + 1;
      if ( meshIdCount != fileNodeCount )
      {
        MDAL::Log::error( MDAL_Status::Err_IncompatibleMesh, name(), "Loading old format failed" );
        return;
      }
    }

In binary format (mdal_binary_dat.cpp, MDAL::DriverBinaryDat::load) this doesn't work, because the comparision is made between the number of values in the dataset (numdata) and the number of nodes in the mesh (vertexCount).

      case CT_NUMDATA:
        // Num data
        if ( read( in, reinterpret_cast< char * >( &numdata ), 4 ) ) return exit_with_error( MDAL_Status::Err_UnknownFormat, "unable to read num data" );
        if ( numdata != static_cast< int >( vertexCount ) ) return exit_with_error( MDAL_Status::Err_IncompatibleMesh, "invalid num data" );
        break;

I think both dataset formats should use the same criteria and for more flexibilty it should be the first one (the maximum node id of the mesh). This would solve the problem of gaps in node numbering. But there may be cases where the node with the maximum ID should be removed from the mesh. In this case loading of datasets would not work, because the criteria is meshIdCount != fileNodeCount and would cause an error. So the only really mandatory check should be: meshIdCount > fileNodeCount where no values can be found in the datasets file for ID greater than fileNodeCount.

A second barrier in file formats can be the number of elements, if stored in datasets files (NC card / CT_NUMCELLS ). Flexibility would rise if this would be ignored completely for vertex based datasets.

I'm aware that the implemented behavior is intended to avoid users from combining mismatching data. I think, the user would recognize this immediately, viewing the results and the benefit of these checks would be less than the higher flexibility and the ability to create clipped TIN-Meshes (e.g. for watersurface elevation).

The NetCDF tests are failing after MDAL is built using GDAL 3.5.0 in Conda-Forge.

The results can be seen here https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=515809&view=logs&j=656edd35-690f-5c53-9ba3-09c10d0bea97&t=e5c8ab1d-8ff9-5cae-b332-e15ae582ed2d&l=2279

packaging mdal on OpenBSD, building against systemwide GTest 1.8.1, i found out that FindGTest.cmake shipped by cmake 3.20.3 defines GTEST_INCLUDE_DIRS (cf https://github.com/Kitware/CMake/blob/master/Modules/FindGTest.cmake#L43) while mdal's tests/CMakeFiles.txt in https://github.com/lutraconsulting/MDAL/blob/master/tests/CMakeLists.txt#L33 uses GTEST_INCLUDE_DIR which is undefined.