Suppose I have a paper with the following text:
Roses are red, violets are blue, sugar is sweet, and so are you [3],[4].
As one can see, the author got the information from two other references.
If I want to add this information in my thesis, do I also have to reference [3] and [4] or can I just reference this particular paper, I got the information from?
If you need a reference that supports the factual information that roses are red etc., then you have to look up the original sources and cite those. If, however, you refer to a synthesis of the information that the author of your sample text has derived from the original sources (for example "flowers have different colors"), then you must cite the sample text.
Having said that, it is sometimes advisable to look up the references also in the latter case. First, because the information contained in them might be interpreted in different ways, and second, because you might learn something useful from them.
Only when you absolutely cannot access the original source, you may resort to a secondary citation, such as [3] as cited in [1].
Similar to @henning's good answer, but emphasizing a way of thinking about such things: be forthright/honest. That is, you did literally learn of [3] and [4] from the secondary source, so it would be helpful to your readers (and factually accurate) to acknowledge this source, as secondary as it may be. That is, you help your readers by explaining the helpful sources you have found. Still, in terms of "primary facts", also [3] and [4] should be cited, for somewhat different reasons.
That is, I think the best (certainly the most honest) scholarship admits what sources were used, how one found things, and so on. To do the opposite, that is, to pretend that one only ever consults "primary sources", is fairly ridiculous, even if it is a popular style. In my field, mathematics, in some cases the original sources are quite inaccessible or anachronistic/archaic, in various senses. So I cannot trust myself to read them with understanding, even if I can acquire a copy. So I must trust others' rewriting, explication, paraphrase, etc. But/and I should acknowledge the historical antecedents even if I cannot directly benefit from them, and also acknowledge accessible/intelligible "secondary" (often with much "value added") sources also.
The process you are describing is called hearsay which is noteably unreliable.I would always go to the original publication if possible. I have seen peer-reviewed review papers that got basic facts wrong. In fact one such paper is accessible here: Applications of shape memory alloys in civil structures. in this paper the authors state without even citing:
In 1932, Chang and Read observed a reversible phase transformation in gold–cadmium (AuCd)...
If you do some initial internet searching, you find nothing because the date and authors are wrong. Once correcting this you find that the 1932 paper was by Ölander and the paper by Chang and Read wasn't published until 1951. As far as truth, the statement blends elements of both papers, so not even that is correct. Therefore heed this as a warning against citing papers you have not read.
I think this depends on the area you are working in. In most sciences the wording of a result is of small importance, so you might say "Flora discovered that roses are red, ... (see [Gardener: A colour guide to flowers, chapter 4.6, ...])". The idea is to tell the reader who did it, and give an available and readable source, e.g. a monograph. This was fine in pre-bibliometrical time, however, today praising an author without citing her is unfair. If Flora's poem is hard to find, but Gardener' colour guide is on every desk, or Flora's original work is difficult to read, you can write Flora[Flora: Poems, ...] discovered ... . We refer the reader to [Gardener] for more information.
Humanities are more subtle. Small changes in the wording or the context can completely alter a statement, so you should not only tell the reader about [3] and[4], but also about the treatment in the secondary paper, which can well have an independent intellectual value.
