The title is the question: can you plagiarize yourself within the same document? For example within a book of thousand pages, do you have to reference ideas from page 50 when you use them on page 950?
The short answer is: No, it is not (self-)plagiarism because it is the same book.
The longer answer is that a reader may want to know more about the subject when reading only the chapter on page 950. It would be highly beneficial for him/her to have a reference to the earlier section on page 50 where more information or more detail about the subject is given.
Finally, you should consider whether it is really necessary to literally repeat yourself within the same book (hint: probably not). This could be an indication of a sub-optimal structure of the book, or maybe providing a short summary and referring to the earlier section for more details is enough.
If the part of concern contains any cited materials, when reused it is better to cite the sources again in case readers had skipped the previous chapter.
If the part of concern is your own words, then for readers' convenience it is better to reference the chapter, section, or page number for easy cross-referencing.
In my opinion, self-plagiaring is more of a misnomer. Plagiarizing is when one presents others' work as if it is his/her own. You cannot steal your own idea. One of the actual concerns is while referring to one's own work, the author could have violated copyright of the publisher that published the work. Since you are referring to another part in the same piece of work, I don't think the copyright violation described here is likely. The other, as one of the commentators pointed out, could be project a false impression on one's productivity.