I am writing a paper, in which I must include a definition of a particular mathematical structure. It's a structure with which very few people would be familiar, thus I wish to include it verbatim, as opposed to citing it. Rewriting it in my own words would only obscure the meaning. Putting a long mathematical definition (perhaps half a page) in quotes looks bizarre and messes up the type setting. Is there a standard way to indicate the definition is cited, whilst avoiding any appearance of plagiarism?
Immediately before the definition, you could say "Quoting verbatim from [source], we define a [whatever-it-is] as follows:"
That is, in general, thinking in terms of honesty/forthrightness is an excellent guide.
This is common practice in mathematics. Definitions are not normally thought of as containing creative content, and if you need to use exactly the same definition as the previous paper, you use the same words. As far as I know, no mathematician considers this to be plagiarism of any kind, provided an appropriate citation is given. Quotation marks or block quotes are not used, because, as you say, it would look weird.
Some phrases you might use:
We define, as in :
The following definition is taken from :
We follow  by defining:
We use the definition from , which we include here for the reader's convenience:
Then format the definition as you normally would. Minor paraphrases and spelling/grammar fixes that do not change the meaning are also fine. If you fix a typo that makes a mathematical difference (e.g. the original author wrote 2 but from context obviously meant 4) then you should point it out explicitly, but this is more for mathematical correctness than citation practice.
Note that if you are publishing in a non-mathematics journal, standards might vary, and you might have to use quotes or otherwise format in some way that would look weird to a mathematician.
If you define terms in a "Definition" environment, then you may state the reference after the heading. For example:
Definition 2 (Noche, 2014). A real number x is said to be small if 0 < x < 1.