I have submitted my paper to a conference A, where it has been presented and is included in the proceedings and is yet to be published online and would be completed in few months. How do I cite this paper in my follow up work to be submitted to another conference B, whose deadlines are way before the previous work can show up online?
The same way everyone cited conference papers 20 years ago.
If the paper is included in the proceedings, as you claim, then presumably the proceedings exists. And since it isn't online, it must be an actual physical book, with a title and a publisher and a copyright date and page numbers and perhaps even an ISBN. So you should cite the paper as part of that physical book.
If, on the other hand, you mean that the paper will be included in the proceedings when the proceedings is eventually published, then follow Buffy's advice: Cite the paper exactly as you would if the proceedings already existed, but with "to appear" instead of the date and DOI.
(To be nice to your reviewers, you might also consider posting a copy of your older conference paper to the arXiv or a similar preprint server, or publishing a technical report version, or otherwise making a copy freely available for them to read.)
Ask the conference chair or, preferably, the proceedings editor how to proceed. They may have already specified what the reference will be. Otherwise, cite the paper as you normally would, but without the date of publication, but naming the venue and "to appear" as part of the citation.
And it may be moot in any case as things may become finalized for the earlier conference before they need to be set in stone for the newer one. The deadline for final copy is normally later than the deadline for submission.