File: org-fold-core.el.html

This file contains library to control temporary invisibility
(folding and unfolding) of text in buffers.

The file implements the following functionality:

- Folding/unfolding regions of text
- Searching and examining boundaries of folded text
- Interactive searching in folded text (via isearch)
- Handling edits in folded text
- Killing/yanking (copying/pasting) of the folded text

To setup folding in an arbitrary buffer, one must call org-fold-core-initialize, optionally providing the list of folding specs to be used in the buffer. The specs can be added, removed, or re-configured later. Read below for more details.

; Folding/unfolding regions of text

User can temporarily hide/reveal (fold/unfold) arbitrary regions or text. The folds can be nested.

Internally, nested folds are marked with different folding specs Overlapping folds marked with the same folding spec are automatically merged, while folds with different folding specs can coexist and be folded/unfolded independently.

When multiple folding specs are applied to the same region of text, text visibility is decided according to the folding spec with topmost priority.

By default, we define two types of folding specs:
- 'org-fold-visible :: the folded text is not hidden
- 'org-fold-hidden :: the folded text is completely hidden

The 'org-fold-visible spec has highest priority allowing parts of text folded with 'org-fold-hidden to be shown unconditionally.

Consider the following Org mode link:
[[file:/path/to/file/file.ext][description]]
Only the word "description" is normally visible in this link.

The way this partial visibility is achieved is combining the two folding specs. The whole link is folded using 'org-fold-hidden folding spec, but the visible part is additionally folded using
'org-fold-visible:

<begin org-fold-hidden>[[file:/path/to/file/file.ext][<begin org-fold-visible>description<end org-fold-visible>]]<end org-fold-hidden>

Because 'org-fold-visible has higher priority than
'org-fold-hidden, it suppresses the 'org-fold-hidden effect and
thus reveals the description part of the link.

Similar to 'org-fold-visible, display of any arbitrary folding spec can be configured using folding spec properties. In particular,
:visible folding spec property controls whether the folded text
is visible or not. If the :visible folding spec property is nil, folded text is hidden or displayed as a constant string (ellipsis) according to the value of :ellipsis folding spec property. See docstring of org-fold-core--specs for the description of all the available folding spec properties.

Folding spec properties of any valid folding spec can be changed any time using org-fold-core-set-folding-spec-property.

If necessary, one can add or remove folding specs using org-fold-core-add-folding-spec and org-fold-core-remove-folding-spec.

If a buffer initialized with org-fold-core-initialize is cloned into indirect buffers, it's folding state is copied to that indirect buffer. The folding states are independent.

When working with indirect buffers that are handled by this library, one has to keep in mind that folding state is preserved on copy when using non-interactive functions. Moreover, the folding states of all the indirect buffers will be copied together.

Example of the implications: Consider a base buffer and indirect buffer with the following state:
----- base buffer --------
* Heading<begin fold>
Some text folded in the base buffer, but unfolded in the indirect buffer<end fold>
* Other heading
Heading unfolded in both the buffers.
---------------------------
------ indirect buffer ----
* Heading
Some text folded in the base buffer, but unfolded in the indirect buffer
* Other heading
Heading unfolded in both the buffers.
----------------------------
If some Elisp code copies the whole "Heading" from the indirect buffer with buffer-substring or match data and inserts it into the base buffer, the inserted heading will be folded since the internal setting for the folding state is shared between the base and indirect buffers. It's just that the indirect buffer ignores the base buffer folding settings. However, as soon as the text is copied back to the base buffer, the folding state will become respected again.

If the described situation is undesired, Elisp code can use filter-buffer-substring instead of buffer-substring. All the folding states that do not belong to the currently active buffer will be cleared in the copied text then. See org-fold-core--buffer-substring-filter for more details.

Because of details of implementation of the folding, it is also not recommended to set text visibility in buffer directly by setting invisible text property to anything other than t. While this should usually work just fine, normal folding can be broken if one sets invisible text property to a value not listed in buffer-invisibility-spec.

; Searching and examining boundaries of folded text

It is possible to examine folding specs (there may be several) of text at point or search for regions with the same folding spec. See functions defined under ";;;; Searching and examining folded text" below for details.

All the folding specs can be specified by symbol representing their name. However, this is not always convenient, especially if the same spec can be used for fold different syntactical structures. Any folding spec can be additionally referenced by a symbol listed in the spec's :alias folding spec property. For example, Org mode's org-fold-outline folding spec can be referenced as any symbol from the following list: '(headline heading outline inlinetask plain-list) The list is the value of the spec's :alias property.

Most of the functions defined below that require a folding spec symbol as their argument, can also accept any symbol from the
:alias spec property to reference that folding spec.

If one wants to search invisible text without using the provided functions, it is important to keep in mind that 'invisible text property may have multiple possible values (not just nil and t). Hence, (next-single-char-property-change pos 'invisible) is not guaranteed to return the boundary of invisible/visible text.

; Interactive searching inside folded text (via isearch)

The library provides a way to control if the folded text can be searchable using isearch. If the text is searchable, it is also possible to control to unfold it temporarily during interactive isearch session.

The isearch behavior is controlled on per-folding-spec basis by setting isearch-open and isearch-ignore folding spec properties. See the docstring of org-fold-core--specs for more details.

; Handling edits inside folded text

The visibility of the text inserted in front, rear, or in the middle of a folded region is managed according to :front-sticky and :rear-sticky folding properties of the corresponding folding spec. The rules are the same with stickiness of text properties in Elisp.

If a text being inserted into the buffer is already folded and invisible (before applying the stickiness rules), then it is revealed. This behavior can be changed by wrapping the insertion code into org-fold-core-ignore-modifications macro. The macro will disable all the processing related to buffer modifications.

The library also provides a way to unfold the text after some destructive changes breaking syntactical structure of the buffer. For example, Org mode automatically reveals folded drawers when the drawer becomes syntactically incorrect:
------- before modification -------
:DRAWER:<begin fold>
Some folded text inside drawer
:END:<end fold>
-----------------------------------
If the ":END:" is edited, drawer syntax is not correct anymore and the folded text is automatically unfolded.
------- after modification --------
:DRAWER:
Some folded text inside drawer
:EN:
-----------------------------------

The described automatic unfolding is controlled by :fragile folding spec property. It's value can be a function checking if changes inside (or around) the fold should drigger the unfold. By default, only changes that directly involve folded regions will trigger the check. In addition, org-fold-core-extend-changed-region-functions can be set to extend the checks to all folded regions intersecting with the region returned by the functions listed in the variable.

The fragility checks can be bypassed if the code doing modifications is wrapped into org-fold-core-ignore-fragility-checks macro.

; Performance considerations

This library is using text properties to hide text. Text properties are much faster than overlays, that could be used for the same purpose. Overlays are implemented with O(n) complexity in Emacs (as for 2021-03-11). It means that any attempt to move through hidden text in a file with many invisible overlays will require time scaling with the number of folded regions (the problem Overlays note of the manual warns about). For curious, historical reasons why overlays are not efficient can be found in https://www.jwz.org/doc/lemacs.html.

Despite using text properties, the performance is still limited by Emacs display engine. For example, >7Mb of text hidden within visible part of a buffer may cause noticeable lags (which is still orders of magnitude better in comparison with overlays). If the performance issues become critical while using this library, it is recommended to minimize the number of folding specs used in the same buffer at a time.

Alternatively, the library provides org-fold-core--optimise-for-huge-buffers for additional speedup. This can be used as a file-local variable in huge buffers. The variable can be set to enable various levels of extra optimization. See the docstring for detailed information.

It is worth noting that when using org-fold-core--optimise-for-huge-buffers with grab-invisible option, folded regions copied to other buffers (including buffers that do not use this library) will remain invisible. org-fold-core provides functions to work around this issue: org-fold-core-remove-optimisation and org-fold-core-update-optimisation, but it is unlikely that a random external package will use them.

Another possible bottleneck is the fragility check after the change related to the folded text. The functions used in :fragile folding properties must be optimized. Also, org-fold-core-ignore-fragility-checks or even org-fold-core-ignore-modifications may be used when appropriate in the performance-critical code. When inserting text from within org-fold-core-ignore-modifications macro, it is recommended to use insert-and-inherit instead of insert and insert-before-markers-and-inherit instead of insert-before-markers to avoid revealing inserted text in the middle of a folded region.

Performance of isearch is currently limited by Emacs isearch implementation. For now, Emacs isearch only supports searching through text hidden using overlays. This library handles isearch by converting folds with matching text to overlays, which may affect performance in case of large number of matches. In the future, Emacs will hopefully accept the relevant patch allowing isearch to work with text hidden via text properties, but the performance hit has to be accepted meanwhile.

Defined variables (20)

Defined functions (49)

Defined faces (0)