When adding the remainder of the cdir string to give the full value of pathout in genPathname(), the current code hardcodes a $TMPDIR string length of 4 characters (src/utils.c, lines 3004-5).

The result of this hardcoding is that the function will re-append any additional characters in the $TMPDIR string beyond "/tmp" into the pathout string when determining its full contents.

This causes failures when trying to open and write to files in $TMPDIR on Debian (and possibly other Linux distributions) that do not use plain /tmp as $TMPDIR for an unprivileged user. On Debian, a regular user's $TMPDIR is /tmp/username, not /tmp.

This commit aims to ensure that the full $TMPDIR string length is used and that only nested path entries below $TMPDIR (and not "/tmp") are appended when determining pathout.

Diagnosed using strace after failures running tesseract-ocr and gscan2pdf. No failures were observed when running as the superuser; strace confirmed the issue was broken pathnames and not r/w privileges.

Hi, Noticed this when attempting to update the Fedora package to 1.75.2 (https://koji.fedoraproject.org/koji/taskinfo?taskID=24954012)

FAIL: webpio_reg
================
################   webpio_reg   ###############
leptonica-1.75.2
libgif 5.1.4 : libjpeg 6b (libjpeg-turbo 1.5.3) : libpng 1.6.34 : libtiff 4.0.9 : zlib 1.2.11 : libwebp 0.6.1
Time to read jpg:   0.000
Time to write webp:   0.005
Time to read webp:   0.000
Time to read jpg:   0.000
Time to write webp:   0.000
Time to read webp:   0.000
Time to read jpg:   0.000
Time to write webp:   0.069
Time to read webp:   0.010
Time to read jpg:   0.009
Time to write webp:   0.515
Time to read webp:   0.049
qual = 50, psnr =  43.501
qual = 75, psnr =  46.071
qual = 90, psnr =  52.379
qual = 100, psnr =  57.460
Lossless write:   0.871 sec
lossless; psnr should be 1000: psnr =  23.222
Failure in webpio_reg: value comparison for index 12
difference = 976.777893 but allowed delta = 0.100000
Time:   3.986 sec
################################################

What steps will reproduce the problem?
1. Download and use the static libraries and include file
2. VS 2012 on Windows 8 Pro 64-Bit
3.

What is the expected output? What do you see instead?
I expect the application to compile but am getting LNK2019 errors on all 
functions defined in allheaders.h

What version of the product are you using? On what operating system?
Windows 8 Pro; Visual studios 2012; Leptonica 

Please provide any additional information below.

Original issue reported on code.google.com by [email protected] on 11 Jul 2013 at 5:27

Hello,

I need to preprocess scanned images in order to improve the following OCR quality. I already able to remove horizontal and vertical lines with Leptonica(Java lept4j) library.

I just wondering if there is a scripts written with Leptonica library in order to do the image preprocessing - for example, cleaning scanning artifacts, noise, etc. Thanks!

First error is a link error: alpha_dec.obj module machine type 'x86' conflicts with target machine type 'x64' in cppan-d-b-d project. Then 10 more errors saying: missing type specifier in tiff.h in the leptonica project

@chewi

I've installed the latest version of automake (etc). Now, when running my autobuild file, I am getting errors:

libtoolize: putting auxiliary files in AC_CONFIG_AUX_DIR, 'config'. libtoolize: copying file 'config/ltmain.sh' libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. libtoolize: copying file 'm4/libtool.m4' libtoolize: copying file 'm4/ltoptions.m4' libtoolize: copying file 'm4/ltsugar.m4' libtoolize: copying file 'm4/ltversion.m4' libtoolize: copying file 'm4/lt~obsolete.m4' configure.ac:38: error: possibly undefined macro: AS_IF If this token and others are legitimate, please use m4_pattern_allow. See the Autoconf documentation. configure.ac:44: error: possibly undefined macro: AC_DEFINE configure.ac:46: error: possibly undefined macro: AC_CHECK_LIB configure.ac:47: error: possibly undefined macro: AC_CHECK_HEADER configure.ac:50: error: possibly undefined macro: AC_SUBST configure.ac:52: error: possibly undefined macro: AC_MSG_ERROR configure.ac:178: error: possibly undefined macro: AC_CHECK_HEADERS configure.ac:179: error: possibly undefined macro: AC_DEFINE_UNQUOTED configure.ac:8: installing 'config/compile' configure.ac:10: installing 'config/missing' prog/Makefile.am: installing 'config/depcomp'

Has something been deprecated?

I could suppress the error messages using m4_pattern_allow([AS_IF]) etc., but could not fix the problem, because when running ./configure, I get this error, and no Makefile is made:

./configure: line 13070: syntax error near unexpected token ZLIB,' ./configure: line 13070: PKG_CHECK_MODULES(ZLIB, zlib,'

On iOS, you can't write to /tmp; each application has its own temporary directory. So we rely on the system to tell us where this directory is located.

The converter relies on the rather constant formatting of the current function's block comments and tries to turn them into proper doxygen format.

The "/*!" doxygen comment introducer and the "\file" tag are inserted when the first line with exact format " * <basename>" is found, e.g. " * adaptmap.c".

The included Doxyfile is a first example for a configuration.

This PR fixes minor deviances in comment format:

• Add missing doxygen comment introducers "/*!".
• Some function's comments had "Return" without a colon ("Return:")
• Many function's comments used @name to reference parameters, while doxygen expects %name for references (@ is a command introducer equal to \).
• Add a new attribute <inout> for parameters in *Destroy() functions and unify the format of the comment.
• Move "Input:" and "Return:" to column 8 where they were in another column
• Replace "Note:" with "Notes:" where appropriate to make conv2doxy correctly detect the detailed description in function's block comments.
• Move "Notes:" to column 4 where it was in another column.
• Replace sub-paragraph " - " with " ~ " to avoid doxygen interpreting the text thereafter as a list entry; i.e. keep pre-formatted indentation.
• Replace "null on error" with "NULL on error" in \return comments
• Replace "Input: stream" with "Input: fp (file stream)" where appropriate.

The idea after merging this pull request is:

• the source tree could or should be tagged with something like Before transition to doxygen comment style.
• conv2doxy.c should be compiled and the binary be run once on every src/*.c file with e.g. ls src/*.c|xargs ./conv2doxy
• now do something like git add src/*.c, git commit -m "After transition to doxygen comment style" and git push.
• finally doxygen can be run and the resulting doc/html be published at some server.
• the output doxygen.log can be used to identify more parameter name mismatches etc.
• whenever new functions are added or existing documentation is fixed or improved, run doxygen again and upload the result. I for example used rsync -avH --del doc/html [email protected]:~/public_html/misc.
• the original, backup files can be removed with rm src/*.c~

This closes #126

Greetings,

I am currently attempting to use Leptonica to dewarp scanned phone book images (e.g., see below).

I wrote a program using the example program prog/dewarp1.c as a template where a model can't successfully be built.

Info in dewarpGetTextlineCenters: finding text line centers
Warning in dewarpRemoveShortLines: lines are relatively short
Warning in dewarpBuildPageModel: linecount 3 < min req'd number of lines (10) for model
Info in dewarpaSetValidModels: page 0: no model successfully built
Info in dewarpaApplyInit: no valid dew model for page 0
Error in dewarpaApplyDisparity: no model available
Info in pixWriteStreamJpeg: converting from 1 to 8 bpp
Error in fopenReadStream: file not found
Error in pixRead: image file not found: /tmp/lept/dewmod/0060.png
Error in pixWrite: pix not defined
Error in fopenReadStream: file not found
Error in pixRead: image file not found: /tmp/lept/dewmod/0070.png
Error in pixWrite: pix not defined
Error in fopenReadStream: file not found
Error in pixRead: image file not found: /tmp/lept/dewapply/002.png
Error in pixWrite: pix not defined
Error in fopenReadStream: file not found
Error in pixRead: image file not found: /tmp/lept/dewapply/003.png
Error in pixWrite: pix not defined

concatenating ... done
pdf file made: /tmp/lept/model/dewarptest1.pdf

Here is a summary of the changes:

• The input image pixs contains the image above
• Excluded converting from RGB to Gray using pixConvertRGBToGray since the image is already in grayscale
• Set the number of dewarp pages from 2 to 1 in dewarpaCreate
• Since the page I'm processing doesn't have a page number, I set the page number from 35 to 0 in dewarpCreate and dewarpaApplyDisparity
• Removed normalizing another image

Here is the full program and image to be processed.

Any help would be greatly appreciated!

What steps will reproduce the problem?
1. .Installed leptonlib-1.67
2.Compiled tesseract 3.00 and installed
3. Ran teserract

What is the expected output? What do you see instead?
Expected tesseract to produce the OCR result.  Got the following errors
Tesseract Open Source OCR Engine with Leptonica
Error in findTiffCompression: function not present
Error in tiffGetCount: function not present
Error reading file ./namefile

What version of the product are you using? On what operating system?
leptonica 1.67 on mac OSX 10.4; tesseract 3.00

Please provide any additional information below.

you can find additional information here:

http://code.google.com/p/tesseract-ocr/issues/detail?id=340#c0

Original issue reported on code.google.com by [email protected] on 4 Jan 2011 at 12:39

What steps will reproduce the problem?
1. it can read gif files but when exit the program halts itself.
2. just occurs this fault with gif files.

What is the expected output? What do you see instead?
gif files in other vision libraries like openCV can be read/write
correctly/ without any fault. giflib4.1.6 is used in openCV and there is no
problem.


What version of the product are you using? On what operating system?
Windows Vista Home Premium.

Please provide any additional information below.
Leptonica v1.65

Original issue reported on code.google.com by [email protected] on 9 May 2010 at 2:48

The function pixWriteMemJpeg uses open_memstream on systems where this available. On other system, the function writes to a temporary file and then reads that in.

libjpeg natively provides a 'memory' destination and 'memory' source. The libjpeg.txt file in the libjpeg source code provides details of this.

For compression, it provides jpeg_mem_dest which takes two arguments .. unsigned char ** outbuffer and unsigned long * outsize

The 2nd argument's data type is different from the one provided in pixWriteMemJpeg which is size_t.

I tried out the compression routine and it works fine .. I copied the existing pixWriteStreamJpeg function, modified the arguments to receive the pdata and psize; and replaced the jpeg_stdio_dest with jpeg_mem_dest.

If you could let me know how you would like this to be implemented, I could do that and submit it.

Thanks K

Good day,

I am looking for a non-AI solution that can detect skew angle of a scanned PDF text document. I have found some solutions that can detect a skew angle from -45 to 45 degrees, but they don't work reliably.

So, I am curious how big of a skew angle can Leptonica detect?

results with defaults 0,NULL,NULL in

The KVK on top left should be left as it was, or at least the bottom half of the V should be one way or the other to be covered by MRC. The text print - wis - clear on the top right should invert.

The applied algorithm might not be sufficiënt. If so DjVu has an expired patented algorithm for foreground/background separation: https://patents.google.com/patent/US6901169

However it performs less when there's noise in the scan that looks like holes in the mask: https://github.com/jwilk/didjvu/issues/21

But in this example, straightly converted from an electronic PDF form, there should be no issue at all.

Before doing OCR i need to detect angle of text, and possibly rotate image but using findSkew function i get incorrect angle, what do I do wrong?

val skewImage: Pix = Binarize.otsuAdaptiveThreshold(Convert.convertTo8(ReadFile.readBitmap(bitmap))) val skew: Float = Skew.findSkew(skewImage) Log.i("skew", skew.toString()

)

Hi maintainer, I see the issue when I use the leptonica. The issue is: "Embedded image in .pdf file is black" Input image: BMP format with spp = 3, d = 32 and has color map (I'm so sorry because I can't provide this image) I used method: l_pixGenerateCIDataForPdf with L_FLATE_ENCODE Do you know this issue?

Detecting this problem has been a bit of detective work. I started from within python3 and wrapped liblept for use with libtesseract in a python3 C-Foreign-Function-Interface.

Background context:

From within python3, I detected a perfect memory leak (100% of data or even more) pushed into the tesseract/lept interface failed to get freed regardless of code changes.

I isolated the python3 interface first to a bare minimum C-interface call, which involved libtesseract and liblept working together. The memleak was present, with plateaus. Either cffi, libtesseract or liblept had a leak, or I was misusing one of the two libraries in cffi such that there was a leak due to my own error.

In order to rule out libtesseract to at least some extent, I set up a C++ api example and tested it for leakage. There was no leakage even though liblept was used as part of this test.

With that done, I dialed in on liblept as the possible source of leakage. When isolating solely the liblept calls, I was able to reproduce a perfect memory leakage with no plateaus.

At this point, I figured that the error was most likely in cffi, since cffi is the most complex component (it bridges two languages), and I posted a bug on cffi, CFFI Issue #527.

Current Issue

Per the advice of one of the developers there, I set up a C code to duplicate the python3 minimal example of leakage.

Between these two codes, there is some flaw... and it is in either liblept, cffi, or (more unlikely, imo) in one of the supporting python3 libraries -- but this does not seem to be the case.

Starting from the Python3 side, I'll try to pinch the issue between working code. Most likely it falls to either lept or cffi, but it is still not clear to me where the bug is exactly. I'll develop my position while writing this issue -- hopefully I'll get some clear information from valgrind on the C-side.

Starting with the detection of the problem in python3:

import gc

import click
import cv2
import numpy
from cffi import FFI
from PIL import Image as ImageModule
from PIL.Image import Image

DEFAULT_DPI = 300


def read_img(img_file: str):
    img = cv2.imread(img_file)
    # favoring numpy's memory management over opencv-python
    img_arr = numpy.copy(img)
    del img
    return img_arr


def cffi_memtest_pix(img: numpy.ndarray, cycles: int = 1):
    """Memory test the image cffi transaction with a standalone cffi interface.

    Run this for an arbitrary number of cycles in a docker container while monitoring the image's memory to confirm
    leakage.

    Args:
        img: the image to test
        cycles: the number of times to load and delete images
    """

    hdr = """
    typedef struct Pix Pix;
    char * getLeptonicaVersion();
    Pix * pixRead(const char* filename);
    Pix * pixCreate(int cols, int rows, int channels);
    Pix * pixSetData(Pix* pix, unsigned int * buffer);
    int   pixSetResolution(Pix * pix, int xres, int yres);
    void pixDestroy(Pix ** pix);
    Pix * pixEndianByteSwap(Pix * pix);
    """
    global DEFAULT_DPI
    ffi = FFI(backend=None)
    ffi.cdef(hdr)
    lept = ffi.dlopen('lept')

    for i in range(cycles):
        pil: Image = ImageModule.fromarray(img).convert("RGBA")
        cols, rows = pil.size
        pil_img_buf = pil.tobytes("raw", "RGBA")

        pix = lept.pixCreate(cols, rows, 32)
        lept.pixSetData(pix, ffi.from_buffer("unsigned int[]", pil_img_buf))

        # disabled for minimal test
        # lept.pixSetResolution(pix, DEFAULT_DPI, DEFAULT_DPI)
        pix = lept.pixEndianByteSwap(pix)  # no-op on big-endian, fixes the problem on little-endian

        _pix = ffi.new("Pix **")
        _pix[0] = pix
        lept.pixDestroy(_pix)
        ffi.release(_pix)

        del pix
        del _pix
        gc.collect()

    ffi.dlclose(lept)


def py_memtest_pix(img: numpy.ndarray, cycles: int = 1):
    """Memory test the python component of the image cffi transaction with a standalone cffi interface.

    Run this for an arbitrary number of cycles in a docker container while monitoring the image's memory to confirm
    leakage.

    Args:
        img: the image to test
        cycles: the number of times to load and delete images
    """

    hdr = """
    typedef struct Pix Pix;
    char * getLeptonicaVersion();
    Pix * pixRead(const char* filename);
    Pix * pixCreate(int cols, int rows, int channels);
    Pix * pixSetData(Pix* pix, unsigned int * buffer);
    int   pixSetResolution(Pix * pix, int xres, int yres);
    void pixDestroy(Pix ** pix);
    Pix * pixEndianByteSwap(Pix * pix);
    """
    global DEFAULT_DPI
    ffi = FFI(backend=None)
    ffi.cdef(hdr)
    lept = ffi.dlopen('lept')

    for i in range(cycles):
        pil: Image = ImageModule.fromarray(img).convert("RGBA")
        pil_img_buf = pil.tobytes("raw", "RGBA")
        gc.collect()

    ffi.dlclose(lept)

@click.command()
@click.argument('img', type=str)
@click.argument('cycles', type=int)
def run(img: str, cycles: int):
    img_ndarr = read_img(img)
    cffi_memtest_pix(img_ndarr, cycles)


if __name__ == "__main__":
    run()

My python3 memory test procedure drops valgrind, since I found it to be problematic for benchmarking python3 memory usage. Instead, I ran python3 code inside of an python3 eval loop running in a docker container. I plotted the docker container's memory usage per second in GNU-plot. X-axis is time in seconds, Y-axis is memory usage in GB.

Python3-only memtest result (PIL + numpy/opencv)

Python3 with CFFI(lept)

C-side analysis (in progress, checking in this issue's content so far):

I'll be starting with this code and I will try to reproduce the leakage. If the leak is not in this library, I'll use this issue as a reference for my issue in cffi:

#include "leptonica/allheaders.h"

int main(){

    struct Pix * r_pix = pixRead("/.../my-large-image.png");

    for (int i = 0; i < 1000; i++){

        // Pix seems to delete the data associated with the original pix when it is set to a different pix
        // so this avoids segfault
        struct Pix * r_pix_cp = pixCopy(NULL, r_pix);
        l_uint32 * pix_data = pixExtractData(r_pix_cp);

        // analog for python3 conversion from PIL
        struct Pix * pix = pixCreate(r_pix->w, r_pix->h, r_pix->d);
        pixSetData(pix, pix_data);
        pixEndianByteSwap(pix);
        pixDestroy(&pix);

        // free the rest of the copied struct
        pixDestroy(&r_pix_cp);
    };

    // free the original pix
    pixDestroy(&r_pix);
}