org-table.el: Enhanced table parsing

* lisp/org-table.el (org-table-to-lisp): Refactored.
* etc/ORG-NEWS: Document changes.

`org-table-to-lisp' is significantly faster.
It no longer uses regexps, nor clobbers the global regexp state.

etc/ORG-NEWS

@@ -1285,6 +1285,12 @@ For symmetry with =\S= and =\sect= for the section symbol, =\P= has
 been added as an another form for the pilcrow symbol currently
 available as =\para=.
 
+*** ~org-table-to-lisp~ no longer clobbers the regexp global state
+
+It does no longer use regexps.
+
+It is also faster. Large tables can be read quickly.
+
 * Version 9.6
 
 ** Important announcements and breaking changes

lisp/org-table.el

@@ -5494,23 +5494,34 @@ for a horizontal separator line, or a list of field values as strings.
 The table is taken from the parameter TXT, or from the buffer at point."
   (if txt
       (with-temp-buffer
+	(buffer-disable-undo)
         (insert txt)
         (goto-char (point-min))
         (org-table-to-lisp))
     (save-excursion
       (goto-char (org-table-begin))
-      (let ((table nil))
-        (while (re-search-forward "\\=[ \t]*|" nil t)
-	  (let ((row nil))
-	    (if (looking-at "-")
-		(push 'hline table)
-	      (while (not (progn (skip-chars-forward " \t") (eolp)))
-		(push (buffer-substring
-		       (point)
-		       (progn (re-search-forward "[ \t]*\\(|\\|$\\)")
-			      (match-beginning 0)))
-		      row))
-	      (push (nreverse row) table)))
+      (let (table)
+        (while (progn (skip-chars-forward " \t")
+                      (eq (following-char) ?|))
+	  (forward-char)
+	  (push
+	   (if (eq (following-char) ?-)
+	       'hline
+	     (let (row)
+	       (while (progn
+                        (skip-chars-forward " \t")
+                        (not (eolp)))
+                 (let ((q (point)))
+                   (skip-chars-forward "^|\n")
+                   (goto-char
+                    (prog1
+                        (let ((p (point)))
+                          (unless (eolp) (setq p (1+ p)))
+                          p)
+	              (skip-chars-backward " \t" q)
+	              (push (buffer-substring-no-properties q (point)) row)))))
+	       (nreverse row)))
+	   table)
 	  (forward-line))
 	(nreverse table)))))