diglib: Calculate the total size of topo file to get the correct off_t_size by HuidaeCho · Pull Request #3163 · OSGeo/grass

HuidaeCho · 2023-09-18T04:57:26Z

off_t sizes for coor and topo files can be different. For example, from an EPSG:4326 location,

> v.mkgrid map=grids_00625deg position=coor coor=-180,-90 box=0.0625,0.0625 grid=2880,5760
> d.mon wx0
> d.vect grids_00625deg
ERROR: Unable to seek: Invalid argument

> ls -al .../vector/grids_00625deg
-rw-r--r-- 1 hcho users 1708966098 Sep 17 22:35 coor
-rw-r--r-- 1 hcho users         95 Sep 17 22:33 dbln
-rw-r--r-- 1 hcho users        173 Sep 17 22:33 head
-rw-r--r-- 1 hcho users        260 Sep 17 22:33 hist
-rw-r--r-- 1 hcho users 6706995200 Sep 17 22:45 sidx
-rw-r--r-- 1 hcho users 2173694570 Sep 17 22:45 topo

coor_size is less than PORT_LONG_MAX (2^31 - 1 = 2147483647; max int), but the topo size is greater than that. Setting ptr->off_t_size = 4 in

grass/lib/vector/diglib/plus_struct.c

Lines 677 to 682 in fd59e94

    
           if (ptr->coor_size > (off_t)PORT_LONG_MAX) { 
        
               /* can only happen when sizeof(off_t) == 8 */ 
        
               ptr->off_t_size = 8; 
        
           } 
        
           else 
        
               ptr->off_t_size = 4;

causes an integer overflow and ptr->Isle_offset is read as -2121341854 in

grass/lib/vector/diglib/plus_struct.c

Line 641 in fd59e94

if (0 >= dig__fread_port_O(&(ptr->Isle_offset), 1, fp, ptr->off_t_size))

This PR calculates the total size of topo file to get the correct off_t_size. Before this PR, the size of coor file was used to determine this offset size for topo file.

…t_size

lib/vector/diglib/plus_struct.c

… and topo files

wenzeslaus · 2023-11-09T20:11:00Z

...ptr->off_t_size is shared between coor and topo files as of now...

Can you please add documentation for these things? Preferably also something in the Doxygen API-level doc.

… of the vector map (OSGeo#3087) * Use gs.find_file() func instead of pygrass module

… drawing items (OSGeo#3162) Device context DrawTex() and DrawLine() method require input x, y parameter argument type as integer type.

Rectangle widget with integer coordinates Offset() method dx, dy parameters require integer argument type.

…rs (OSGeo#3168) Addon module name start with db.*, ps.*, r3.*, wx.*, during installation on the OS MS Windows platform.

Compatible with Matplotlib v3.

* Translated using Weblate (Russian) Currently translated at 100.0% (2193 of 2193 strings) Translation: GRASS GIS/grasslibs Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasslibs/ru/ * Translated using Weblate (Russian) Currently translated at 97.6% (4277 of 4380 strings) Translation: GRASS GIS/grasswxpy Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasswxpy/ru/ * Translated using Weblate (Russian) Currently translated at 0.9% (121 of 12647 strings) Translation: GRASS GIS/grassmods Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grassmods/ru/ * Translated using Weblate (German) Currently translated at 40.0% (879 of 2197 strings) Translation: GRASS GIS/grasslibs Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasslibs/de/ * Translated using Weblate (Russian) Currently translated at 100.0% (2193 of 2193 strings) Translation: GRASS GIS/grasslibs Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasslibs/ru/ * Translated using Weblate (German) Currently translated at 34.4% (4394 of 12767 strings) Translation: GRASS GIS/grassmods Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grassmods/de/ * Translated using Weblate (Russian) Currently translated at 100.0% (4380 of 4380 strings) Translation: GRASS GIS/grasswxpy Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasswxpy/ru/ * Translated using Weblate (Russian) Currently translated at 1.4% (182 of 12647 strings) Translation: GRASS GIS/grassmods Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grassmods/ru/ * Translated using Weblate (German) Currently translated at 34.4% (4403 of 12767 strings) Translation: GRASS GIS/grassmods Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grassmods/de/ * Translated using Weblate (German) Currently translated at 40.2% (885 of 2197 strings) Translation: GRASS GIS/grasslibs Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasslibs/de/ * Translated using Weblate (German) Currently translated at 61.9% (2734 of 4415 strings) Translation: GRASS GIS/grasswxpy Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasswxpy/de/ --------- Co-authored-by: Sergey Kazorin <[email protected]> Co-authored-by: Markus <[email protected]> Co-authored-by: Weblate <[email protected]>

Currently translated at 22.3% (489 of 2187 strings) Translation: GRASS GIS/grasslibs Translate-URL: https://weblate.osgeo.org/projects/grass-gis/grasslibs/ko/ Co-authored-by: Huidae Cho <[email protected]>

…OSGeo#3189) Remove vertical sizer Add() method flags parameter argument wx.ALIGN_CENTER_VERTICAL flag. Only horizontal alignment flags can be used in vertical sizers.

…eo#3188) If the Graphical Modeler window is closed.

…SGeo#3177) * wxGUI/main_window: add and use custom SingleWindowAuiManager class Which override base OnClose window close event handler method to prevent prematurely uninitialize manager. wxWidgets/Phoenix#2460 * Update the package documentation class list

OSGeo#3143)

…SGeo#3089)

…e app (OSGeo#2456) `g.gui.vdigit` module.

* wxGUI/psmap: fix wx.adv.PseudoDC() class DrawRotatedText() method x, y param arg int type

HuidaeCho · 2024-01-09T16:57:35Z

Oh! I messed it up... Closing it for now.

HuidaeCho · 2024-01-09T17:46:15Z

...ptr->off_t_size is shared between coor and topo files as of now...

Can you please add documentation for these things? Preferably also something in the Doxygen API-level doc.

Added it in #3351

diglib: Calculate the total size of topo file to get the correct off_…

a5047c5

…t_size

HuidaeCho added vector Related to vector data processing bug Something isn't working C Related code is in C backport to 8.3 labels Sep 18, 2023

metzm reviewed Sep 19, 2023

View reviewed changes

lib/vector/diglib/plus_struct.c Show resolved Hide resolved

Try a coor_size check first because off_t_size is shared between coor…

48ac50b

… and topo files

HuidaeCho requested a review from metzm September 19, 2023 22:27

neteler added this to the 8.4.0 milestone Oct 5, 2023

landam assigned HuidaeCho Nov 19, 2023

tmszi and others added 18 commits January 9, 2024 09:54

wxGUI/psmap: fix vector map properties dialog correct typing the name…

8822930

… of the vector map (OSGeo#3087) * Use gs.find_file() func instead of pygrass module

wxGUI/psmap: fix vector line map props dialog PenStyleComboBox widget…

b350521

… drawing items (OSGeo#3162) Device context DrawTex() and DrawLine() method require input x, y parameter argument type as integer type.

wxGUI/psmap: fix simple point graphic recalculate position (OSGeo#3164)

806faec

Rectangle widget with integer coordinates Offset() method dx, dy parameters require integer argument type.

g.extension: fix registration addon with module name start with 2 cha…

cc6cd02

…rs (OSGeo#3168) Addon module name start with db.*, ps.*, r3.*, wx.*, during installation on the OS MS Windows platform.

wxGUI/iclass: fix showing scatter plots (OSGeo#3169)

242c3a3

Compatible with Matplotlib v3.

wxGUI/image2target: fix showing Ground Control Points settings dialog (…

f86d408

…OSGeo#3189) Remove vertical sizer Add() method flags parameter argument wx.ALIGN_CENTER_VERTICAL flag. Only horizontal alignment flags can be used in vertical sizers.

wxGUI/gmodeler: fix storing current model settings to model file (OSG…

63f403f

…eo#3188) If the Graphical Modeler window is closed.

wxGUI/preferences: fix setting command output font (OSGeo#3187)

83b7705

wxGUI/iclass: fix zoom to scatter plot extend (OSGeo#3184)

5168502

wxGUI/iclass: fix closing scatter plot pane via x button (OSGeo#3172)

663a118

wxGUI/dbmgr: do not modify layer settings if settings have not changed (

e76abea

OSGeo#3143)

wxGUI/nviz: make tools panel verically and horizontally scrollable (O…

7720245

…SGeo#3089)

wxGUI/vdigit: fix close frame if Vector Digitizer is run as standalon…

10a8ff9

…e app (OSGeo#2456) `g.gui.vdigit` module.

wxGUI/psmap: fix showing image map element preview (OSGeo#3193)

6b01841

wxGUI/mapswipe: fix saving display to file (OSGeo#3185)

43625fa

* wxGUI/psmap: fix wx.adv.PseudoDC() class DrawRotatedText() method x, y param arg int type

github-actions bot added RFC Request For Comment (RFC) document libraries module docs markdown Related to markdown, markdown files labels Jan 9, 2024

HuidaeCho force-pushed the diglib_topo_offset branch from d5bd14a to 4ab7ea9 Compare January 9, 2024 16:55

HuidaeCho closed this Jan 9, 2024

HuidaeCho deleted the diglib_topo_offset branch January 9, 2024 16:57

HuidaeCho mentioned this pull request Jan 9, 2024

lib/diglib: calculate the total size of topo file to get the correct off_t_size #3351

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Comments

diglib: Calculate the total size of topo file to get the correct off_t_size#3163

diglib: Calculate the total size of topo file to get the correct off_t_size#3163
HuidaeCho wants to merge 119 commits intoOSGeo:mainfrom
HuidaeCho:diglib_topo_offset

HuidaeCho commented Sep 18, 2023

Uh oh!

Uh oh!

wenzeslaus commented Nov 9, 2023

Uh oh!

HuidaeCho commented Jan 9, 2024

Uh oh!

HuidaeCho commented Jan 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants

	if (ptr->coor_size > (off_t)PORT_LONG_MAX) {
	/* can only happen when sizeof(off_t) == 8 */
	ptr->off_t_size = 8;
	}
	else
	ptr->off_t_size = 4;

Uh oh!

Comments

Conversation

HuidaeCho commented Sep 18, 2023

Uh oh!

Uh oh!

wenzeslaus commented Nov 9, 2023

Uh oh!

HuidaeCho commented Jan 9, 2024

Uh oh!

HuidaeCho commented Jan 9, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

20 participants