Access Document Scanners in Java

Sep 22, 2023

 Document Scanning

•  REST API
•  JAVA
•  WEB TWAIN
•  JAVAFX

Dynamic Web TWAIN is an SDK which enables document scanning from browsers. Under its hood, a backend service named Dynamic Web TWAIN Service is running to communicate with scanners via protocols like TWAIN, WIA, eSCL, SANE and ICA. The service runs on Windows, macOS and Linux.

Starting from Dynamic Web TWAIN v18.4, Dynamic Web TWAIN Service will be accessible via REST APIs so that we can use different programming languages to create document scanning applications.

In this article, we are going to talk about how to access document scanners using the REST APIs in Java. A desktop app is built using JavaFX.

Some advantages of using the REST APIs of Dynamic Web TWAIN Service for document scanning in Java:

1. It is cross-platform supporting multiple scanning protocols.
2. If we directly call TWAIN API, we have to use JRE 32-bit as most drivers are 32-bit. Using the REST APIs does not have this problem.

This article is Part 3 in a 7-Part Series.

• Part 1 - Dynamic Web TWAIN Service REST API - Scan Documents in Node.js
• Part 2 - Building a Document Digitization App with Flutter for TWAIN, WIA, and eSCL Scanners
• Part 3 - Access Document Scanners in Java
• Part 4 - How to Scan Documents from TWAIN, WIA, SANE Compatible Scanners in Python
• Part 5 - Building .NET Apps for Scanning Documents from TWAIN, WIA, SANE, and eSCL Scanners
• Part 6 - Scanning Documents to Web Pages with JavaScript and Dynamic Web TWAIN RESTful API
• Part 7 - How to Build a Remote Document Scanner in SwiftUI to Digitize Documents Over Network

Prerequisites

• A license for Dynamic Web TWAIN is needed. You can apply for a license here.
• You need to install Dynamic Web TWAIN Service on your device. You can find the download links in the following table:

Platform	Download Link
Windows	Dynamsoft-Service-Setup.msi
macOS	Dynamsoft-Service-Setup.pkg
Linux	Dynamsoft-Service-Setup.deb Dynamsoft-Service-Setup-arm64.deb Dynamsoft-Service-Setup-mips64el.deb Dynamsoft-Service-Setup.rpm

Overview of the REST API

Endpoint: http://127.0.0.1:18622. You can configure it by visiting http://127.0.0.1:18625/.

APIs:

1. List scanners.

   HTTP method and URL: GET /DWTAPI/Scanners

   Sample response:

   [
     {
       "name":"scanner name",
       "device":"detailed info of the scanner",
       "type": 16
     }
   ]
   

   The following is a list of scanner types and their corresponding values.

   16: TWAIN
   32: WIA
   64: TWAINX64
   128: ICA
   256: SANE
   512: eSCL
   1024: WIFIDIRECT
   2048: WIATWAIN
   

2. Create a document scanning job.

   HTTP method and URL: POST /DWTAPI/ScanJobs

   Sample request body:

   {
     "license":"license of Dynamic Web TWAIN",
     "device":"detailed info of the scanner", #optional. Use the latest device by default
     "config":{ # Device configuration https://www.dynamsoft.com/web-twain/docs/info/api/Interfaces.html#DeviceConfiguration (optional)
       "IfShowUI":true, # show the UI of the scanner
       "Resolution":200,
       "IfFeederEnabled":false, # enable auto document feeder
       "IfDuplexEnabled":false # enable duplex document scanning
     },
     "caps":{ # Capabilities https://www.dynamsoft.com/web-twain/docs/info/api/Interfaces.html#capabilities (optional)
       "exception":"ignore",
       "capabilities":[
         {
           "capability":"", #pixel type
           "curValue":0 #0: black&white, 1: gray, 2: color
         }
       ]
     }
   }
   

   Response:

   201 status code with the job ID as the response body

3. Retrieve a scanned document image.

   HTTP method and URL: GET /DWTAPI/ScanJobs/:jobid/NextDocument

   Response:

   200 with the bytes of the image

4. Get the info of a scanning job.

   HTTP method and URL: GET /DWTAPI/ScanJobs/:jobid/DocumentInfo

5. Delete a scanning job.

   HTTP method and URL: DELETE /DWTAPI/ScanJobs/:jobid

New JavaFX Project

Create a new JavaFX project using IntelliJ IDEA.

Add Dependencies

Add OKHttp as the HTTP library in pom.xml. OKHttp works for both desktop and Android platforms.

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>4.11.0</version>
</dependency>

Add Jackson as the JSON library in pom.xml.

<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-core</artifactId>
    <version>2.15.2</version>
</dependency>
<dependency>
    <groupId>com.fasterxml.jackson.core</groupId>
    <artifactId>jackson-databind</artifactId>
    <version>2.15.2</version>
</dependency>

Also, add PDFBox for saving the scanned document as a PDF file.

<dependency>
    <groupId>org.apache.pdfbox</groupId>
    <artifactId>pdfbox</artifactId>
    <version>3.0.0</version>
</dependency>

Create Classes representing the Data

1. Scanner.

   public class Scanner {
       public String name;
       public int type;
       public String device;
       public Scanner(String name, int type, String device){
           this.name = name;
           this.type = type;
           this.device = device;
       }
   }
   

2. Device type constants.

   public class DeviceType {
       public static final int TWAIN = 16;
       public static final int WIA = 32;
       public static final int TWAINX64 = 64;
       public static final int ICA = 128;
       public static final int SANE = 256;
       public static final int ESCL = 512;
       public static final int WIFIDIRECT = 1024;
       public static final int WIATWAIN = 2048;
       public  static String getDisplayName(int type) throws Exception {
         if (type == TWAIN) {
             return "TWAIN";
         }else if (type == WIA) {
             return "WIA";
         }else if (type == TWAINX64) {
             return "TWAINX64";
         }else if (type == ICA) {
             return "ICA";
         }else if (type == SANE) {
             return "SANE";
         }else if (type == ESCL) {
             return "ESCL";
         }else if (type == WIFIDIRECT) {
             return "WIFIDIRECT";
         }else if (type == WIATWAIN) {
             return "WIATWAIN";
         }else{
             throw new Exception("Invalid type");
         }
       }
   }
   

3. Device configuration.

   public class DeviceConfiguration {
       public boolean IfShowUI = false;
       public int Resolution = 200;
       public boolean IfFeederEnabled = false;
       public boolean IfDuplexEnabled = false;
   }
   

4. Capability setup for one scanner capability.

   public class CapabilitySetup {
       public int capability;
       public Object curValue;
       public String exception = "ignore";
   }
   

5. Capabilities.

   public class Capabilities {
       public String exception = "";
       public List<CapabilitySetup> capabilities = new ArrayList<CapabilitySetup>();
   }
   

Dynamic Web TWAIN Service Class

Create a new Dynamic Web TWAIN Service for calling the REST APIs.

1. Create a basic class with the following content.

   public class DynamsoftService {
       private String endPoint = "http://127.0.0.1:18622";
       private String license = "";
       public DynamsoftService(){
   
       }
       public DynamsoftService(String endPoint, String license){
           this.endPoint = endPoint;
           this.license = license;
       }
   }
   

2. Add a getScanners method to get the list of scanners.

   public List<Scanner> getScanners() throws IOException, InterruptedException {
       OkHttpClient client = new OkHttpClient();
       Request request = new Request.Builder()
               .url(endPoint+"/DWTAPI/Scanners")
               .build();
       try (Response response = client.newCall(request).execute()) {
           String body = response.body().string();
           List<Scanner> scanners = new ArrayList<Scanner>();
           ObjectMapper objectMapper = new ObjectMapper();
           List<Map<String,Object>> parsed = objectMapper.readValue(body,new TypeReference<List<Map<String,Object>>>() {});
   
           for (Map<String,Object> item:parsed) {
               int type = (int) item.get("type");
               String name = (String) item.get("name");
               String device = (String) item.get("device");
               Scanner scanner = new Scanner(name,type,device);
               scanners.add(scanner);
           }
           return scanners;
       }
   }
   

3. Add a createScanJob method to create a scanning job.

   public String createScanJob(Scanner scanner) throws Exception {
       return createScanJob(scanner,null,null);
   }
   
   public String createScanJob(Scanner scanner,DeviceConfiguration config,Capabilities capabilities) throws Exception {
       Map<String,Object> body = new HashMap<String,Object>();
       body.put("license",this.license);
       body.put("device",scanner.device);
       if (config != null) {
           body.put("config",config);
       }
       if (capabilities != null) {
           body.put("caps",capabilities);
       }
       ObjectMapper objectMapper = new ObjectMapper();
       String jsonBody = objectMapper.writeValueAsString(body);
   
       OkHttpClient client = new OkHttpClient.Builder()
               .connectTimeout(120, TimeUnit.SECONDS)
               .build();
       RequestBody requestBody = RequestBody.create(jsonBody, JSON);
       Request request = new Request.Builder()
               .url(endPoint+"/DWTAPI/ScanJobs?timeout=120")
               .post(requestBody)
               .build();
       try (Response response = client.newCall(request).execute()) {
           if (response.code() == 201) {
               return response.body().string();
           }else{
               throw new Exception(response.body().string());
           }
       }
   }
   

4. Add a nextDocument method to get the document image.

   public byte[] nextDocument(String jobID) throws Exception {
       return getImage(jobID);
   }
   
   private byte[] getImage(String jobID) throws Exception {
       OkHttpClient client = new OkHttpClient.Builder()
               .connectTimeout(120, TimeUnit.SECONDS)
               .build();
       Request request = new Request.Builder()
               .url(endPoint+"/DWTAPI/ScanJobs/"+jobID+"/NextDocument?timeout=120")
               .build();
       String body = "";
       try (Response response = client.newCall(request).execute()) {
           if (response.code() == 200) {
               return response.body().bytes();
           }else{
               return null;
           }
       }
   }
   

Update the Stage as a Document Scanner

Next, we can update the primary stage to make it work as a document scanner.

First, we can design the layout of the FXML file with SceneBuilder. On the left, there are controls to configure the scanning and on the right, there is a ListView for displaying the scanned images.

Then, in the controller, implement relevant events and initializations.

1. Load the lists of scanners, resolutions and pixel types in ComboBoxes in the initialization process.

   public void initialize(){
       this.loadResolutions();
       this.loadPixelTypes();
       this.loadScanners();
   }
   
   private void loadResolutions(){
       List<Integer> resolutions = new ArrayList<Integer>();
       resolutions.add(100);
       resolutions.add(200);
       resolutions.add(300);
       resolutionComboBox.setItems(FXCollections.observableList(resolutions));
       resolutionComboBox.getSelectionModel().select(1);
   }
   
   private void loadPixelTypes(){
       List<String> pixelTypes = new ArrayList<String>();
       pixelTypes.add("Black & White");
       pixelTypes.add("Gray");
       pixelTypes.add("Color");
       pixelTypeComboBox.setItems(FXCollections.observableList(pixelTypes));
       pixelTypeComboBox.getSelectionModel().select(0);
   }
   
   private void loadScanners() throws IOException, InterruptedException {
       scanners = service.getScanners();
       List<String> names = new ArrayList<String>();
       for (Scanner scanner:scanners) {
           try {
               names.add(scanner.name + " (" +DeviceType.getDisplayName(scanner.type)+ ")");
           } catch (Exception e) {
               System.out.println(e.getMessage());
           }
       }
       scannersComboBox.setItems(FXCollections.observableList(names));
       if (names.size()>0) {
           scannersComboBox.getSelectionModel().select(0);
       }
   }
   

2. Define a DocumentImage class for the cells of the ListView. The cell of the ListView will contain an ImageView.

   public class DocumentImage {
       public ImageView imageView;
       public byte[] image;
       public DocumentImage(ImageView imageView,byte[] image) {
           this.imageView = imageView;
           this.image = image;
       }
   }
   

3. Update the ListView’s cell factory so that it displays an ImageView.

   documentListView.setCellFactory(param -> new ListCell<DocumentImage>() {
       {
           prefWidthProperty().bind(documentListView.widthProperty().subtract(30));
           setMaxWidth(Control.USE_PREF_SIZE);
       }
       @Override
       protected void updateItem(DocumentImage item, boolean empty) {
           super.updateItem(item, empty);
   
           if (empty) {
               setGraphic(null);
           } else {
               item.imageView.setFitWidth(documentListView.widthProperty().subtract(30).doubleValue());
               setGraphic(item.imageView);
           }
       }
   });
   

4. Scan documents after the scan button is clicked. The images will be displayed in the ListView.

   @FXML
   protected void onScanButtonClicked() {
       int selectedIndex = scannersComboBox.getSelectionModel().getSelectedIndex();
       if (selectedIndex != -1) {
           progressStage.show();
           Thread t = new Thread(() -> {
               Scanner scanner  = scanners.get(selectedIndex);
               try {
                   DeviceConfiguration config = new DeviceConfiguration();
                   config.IfShowUI = showUICheckBox.isSelected();
                   config.IfDuplexEnabled = duplexCheckBox.isSelected();
                   config.IfFeederEnabled = ADFCheckBox.isSelected();
                   config.Resolution = (int) resolutionComboBox.getSelectionModel().getSelectedItem();
                   Capabilities caps = new Capabilities();
                   caps.exception = "ignore";
                   caps.capabilities = new ArrayList<CapabilitySetup>();
                   CapabilitySetup pixelTypeSetup = new CapabilitySetup();
                   pixelTypeSetup.capability = 257;
                   pixelTypeSetup.curValue = pixelTypeComboBox.getSelectionModel().getSelectedIndex();
                   caps.capabilities.add(pixelTypeSetup);
                   String jobID = service.createScanJob(scanner,config,caps);
                   System.out.println("ID: "+jobID);
                   byte[] image = service.nextDocument(jobID);
                   while (image != null){
                       loadImage(image);
                       image = service.nextDocument(jobID);
                   }
               } catch (Exception e) {
                   System.out.println(e.getMessage());
               }
               Platform.runLater(() -> {
                   progressStage.close();
               });
           });
           t.start();
       }
   }
      
   private void loadImage(byte[] image){
       Image img = new Image(new ByteArrayInputStream(image));
       ImageView iv = new ImageView();
       iv.setPreserveRatio(true);
       iv.setImage(img);
       DocumentImage di = new DocumentImage(iv,image);
       documentListView.getItems().add(di);
   }
   

5. Listen to the changes of the width of the ListView. When the width changes, update the ImageViews’ size.

   ChangeListener<Number> changeListener = new ChangeListener<Number>() {
       @Override
       public void changed(ObservableValue<? extends Number> observable, Number oldValue, Number newValue) {
           for (DocumentImage item:documentListView.getItems()) {
               item.imageView.setFitWidth(documentListView.widthProperty().subtract(30).doubleValue());
           }
       }
   };
   documentListView.widthProperty().addListener(changeListener);
   

6. Add a context menu for the ListView for deleting selected document images.

   ContextMenu contextMenu = new ContextMenu();
   MenuItem deleteMenuItem = new MenuItem("Delete selected");
   deleteMenuItem.setOnAction(e -> {
       var indices = documentListView.getSelectionModel().getSelectedIndices();
       for (int i = indices.size() - 1; i >= 0; i--) {
           int index = indices.get(i);
           documentListView.getItems().remove(index);
       }
   });
   contextMenu.getItems().add(deleteMenuItem);
   

7. Use PDFBox to save the scanned document into a PDF file.

   @FXML
   protected void onSaveButtonClicked() throws IOException {
       FileChooser fileChooser = new FileChooser();
       fileChooser.setTitle("Open Resource File");
       File fileToSave = fileChooser.showSaveDialog(null);
       if (fileToSave != null) {
           PDDocument document = new PDDocument();
           int index = 0;
           for (DocumentImage di: documentListView.getItems()) {
               index = index + 1;
               ImageView imageView = di.imageView;
               PDRectangle rect = new PDRectangle((float) imageView.getImage().getWidth(),(float) imageView.getImage().getHeight());
               System.out.println(rect);
               PDPage page = new PDPage(rect);
               document.addPage(page);
               PDPageContentStream contentStream = new PDPageContentStream(document, page);
               PDImageXObject image
                       = PDImageXObject.createFromByteArray(document,di.image,String.valueOf(index));
               contentStream.drawImage(image, 0, 0);
               contentStream.close();
           }
           document.save(fileToSave.getAbsolutePath());
           document.close();
       }
   }
   

All right, we have now finished writing the demo app.

Source Code

Get the source code of the demo to have a try:

https://github.com/tony-xlh/JavaFX-Document-Scanner

You can include the library in your project via Jitpack: https://jitpack.io/#tony-xlh/docscan4j