The denote-open-link-function user option

The user option denote-open-link-function specifies the function used by Denote to open the file of a link. The default value opens the file in the other window. Another common value is the function find-file, which will open the file in the current window. Users may also specify a function of their choosing.

Note that this is relevant in buffers other than Org mode because Org has its own mechanism for how to open links (read the documentation of the command org-open-at-point).