The first part isn't a question but I'm gonna answer it anyway.
Essentially: Yes. I'm not hugely well-versed in my Next Gen fic but I've done my dabbling and I, too, noticed all of the clichés that you did, the common tropes invoked. They were obviously there for a reason, there was obviously something which appealed about these elements, and appealed to a huge crowd, I just didn't like the way they'd often been done.
So, being an arrogant sort, I thought, "Let's do this right." Deconstruct what would really happen with ScoRose (because I really doubt they'd get disowned by their parents and the like for it), or what would really happen with a Slytherin Albus who's Scorp's best mate, or why an uptight Rose would really be so uptight. Despite the above I'm sometimes protective of clichés because what people often dislike isn't the cliché themselves, it's seeing something done badly or unimaginatively over and over.
But now, er, on with actual questions.
1) Number #1 uninvoked challenge of Next Gen is the huge freaking swathe of the extended Weasley clan in that generation. They're all there and I would imagine a lot of the family would be pretty close, and yet how do you handle something like a dozen kids without drowning in ginger? Kids whose backgrounds and upbringing are pretty well-accounted for, whose parents' personalities are quite accounted for. Of course my answer was, "I don't," but they're still in my head for the Stygian Trilogy. Especially James, Lily, and Hugo. Frederick the Younger is somewhere in my brain rattling about. But it was an argument for a plot device like Phlegethon - it got rid of them! I didn't have to worry about them!
Of the stuff you've mentioned: It's Hard. I suppose for the sake of a story I'm already making Draco out to be a worse human being and father than I legitimately think he would be in the canon, but I still wanted to keep it believable. Which meant that Rose and Albus had to have loving and stable and probably affluent upbringings, which for sixteen year-olds starts to already shove them in certain holes (especially when you're playing with archetypal roles). Rose was easier, because Scorpius came first as a character in my head and Rose came soon after, created very much to be his Belligerent Sexual Tension foil. Though she was someone I was really worried about as the first few chapters of Ignite are not kind to her - Chapter 4 wasn't meant to have a segment from her POV but it was apparent it was needed, and needed badly.
Albus has been the monster to write. To make him like his father and yet not, and without invoking any of the truly annoying clichés like being hated by his family for being in Slytherin (I just don't find it believable) or feeling overshadowed by his parents' achievements and trying to prove himself - when he was meant to be the down-to-earth counterpart for Scorpius. It's why he's been one of the weaker of the characters in the Trilogy; for a time he's been a bit neither fish nor fowl and thus slightly bland. The good news is that as the stakes are getting higher, I'm getting a handle on him and his morality and where he fits in with everyone.
There has, however, been no 'trick'. I don't sit down and list their parents' qualities and flaws and then jumble them up. That's not how I do character building; they come into my brain and we have a chat and figure out what needs changing and what works. Which is why Rose, who was pretty chatty, has a lot of her parents in her while not being a carbon copy of Hermione, and Albus, who was less chatty, was slower to cooperate.
2) Would you believe I've been trying to 'quit' fanfiction for about 3 years now? I finished Shade to Shade and was like, "Well, I have all of these ideas on what happens during Book 7, but I've got a manuscript to write." But even though 'proper' writing began, six months later came November and I needed a break from agonising over every single word and said, "I'll write Falls the Shadow for NaNoWriMo. It'll be fine." And I wrote 50k words of it and burnt myself out, and then had a year of being so unproductive it's not even funny. Crawled my way eventually to finishing the manuscript and found myself full of vim and vigour and thought, "I'll finish Shadow while I'm editing!" And I did, and the ideas kept rolling because I was just in such a creative headspace, and the gang's tale wasn't finished with Shadow and I had ideas for Beyond This Place. And I wrote that in three months, and then Ignite came along as a notion - as a bloody Trilogy - and I succumbed. My current plan is to finish Starfall (no mean feat), finish the untitled trilogy ending, and then leave fanfiction. Despite that I'm still writing the odd short story for the Anguisverse which might see the light of day Soon.
But aside from the joke of being 'trapped' - I hate leaving projects unfinished - it's actually a great way to blow off steam. Perhaps I'm just not yet confident enough in my original fiction, but it takes a lot of hard work, a lot of strict planning, while I feel I can dive into fanfiction and rattle it off with a broad, scribbled scheme. I feel much more comfortable just hitting Wiki for a bit and producing the research behind Starfall while I'd probably have to spend hours in the library before I'd feel comfortable presenting that kind of a mythical/real world story for original fiction. Even if the end result would be much the same. Fanfiction has lower expectations (though I like to exceed them), less pressure, and I continuously get feedback and support by posting in a serialised form. When I'm up to my elbows in editing prose where I have to scourge all adverbs off the face of the world, and despising every word I'm penning, this is a serious boost to my motivation. "Of course I'm good, and good enough to make it," I tell myself. "People are enjoying Starfall, see?"
But no, I don't intend to stay in fandom when the Stygian Trilogy ends. Considering I'll be happy if Starfall is done by the end of the year and it'd probably be another year after that for the third, this is a pretty loose deadline.
3) I write, above all, for my own enjoyment. I love, of course, when other people enjoy what I'm writing, but it's rare for me to sit down and go, "I'm going to write a crowd-pleaser." Renaissance I actually did write to try to lure in some readers who might go on to read the Anguisverse. But I'm a character writer first and foremost; the concepts which spring into my brain, the plot bunnies, are rarely, "This happens here! Hilarity ensues!" It's usually a, "This sort of person is faced with this sort of challenge. How do they handle it?" And often that lends itself better to OCs than it does to canon characters. The biggest challenge is people not reading what I write. It can be very disheartening, and the Anguisverse only really took off because I was so invested in these guys I wanted to write it for me.
Otherwise the 'challenges' are more like fun. I get to build entire social groups with friends, enemies, lovers, mentors, all within a pre-existing setting. I get to go, "I wonder what would happen if I played with this element of JK's setting and this idea" and then create a character to match it. It's harder work because I don't have my relationship dynamics out-of-the-box like with fanfic of canon characters, but I find this entertaining, not an obstacle.
I suppose the big 'trick', or advice I'd give, would be to make the OCs relevant. Make them fill a niche which needs exploring, which people want to learn about, without them overshadowing any canon characters or their achievements. Next Gen really are the best of both worlds, though, which is why I was lured into it - they're OCs. At best we have guesses and fanon conventions of what they're like, but we don't know, at all. And yet people lap up Next Gen like nobody's business. So people are, actually, willing to read Original Characters. You just have to present it in the right way.
4) Matt takes more after Jen when it comes to being studious. However Jen Riley is patient and empathetic, and Matt's not hugely good on either. When Matt knows how to say just the right thing to upset someone, that's his father coming through. When he broods through a problem instead of tackling it, that's his father coming through. Gabriel wasn't really more grown-up, he was just more isolated. He was lacking some basic fundamental issues of how to relate to people and cope socially, and just hid it all through a veneer of smug superiority.
He will get over Rose. I have plans for the boy. Then again, I wanted to write a Triangle which wasn't sorted out by Murdering the Hypotenuse (Tanith-Tobias-Annie, Gabriel-Jen-Nick). It'll be a long and awkward road and that's what I want to explore.
But I'm glad you love him, because he's my not-so-secret favourite of Starfall.
5) I considered not answering this for Reasons. But I won't be so cruel. Tobias and Tanith are married and still together. They had kids a bit later than Gabe and Jen, I think they'd hit at least 30 before they started to. Tobias had left politics to return to journalism and so had the opportunity to take care of the kids and work, which was the problem beforehand - neither of them wanted to compromise their work for the children. Jen and Gabe's youngest daughter is in the same year as Tobias and Tanith's son, their eldest. I believe both Robert Grey (middle name Altair) and Sophie Doyle are Slytherins.
But the Anguisverse will ride again. I have all sorts of half-finished scribbles which I need to turn into finished scribbles, and seeing as my plans are nothing more involved than something in the Latet vein, an anthology of non-linear stories of The Rest of Their Lives written as I please, it's not an enormous commitment. But there are scenes I want to cover, incidents of their lives, including perhaps some insight into what they got up to during the Phlegethon Crisis. I also really want to write some stuff about Jen and the Lions and Gabe I skipped during Shadow, which is silly - but they're the most under-developed relationship and characters of the lot, so I wouldn't be surprised if the lion's share of focus went on them. Hopefully I'll have something Soon.
Though I really don't know how to write stories of That Funny Thing That Happened When Their Kid Was Five Years Old On a Summer's Day, so people are just going to have to assume that kind of Happily Ever After stuff happened.
Thanks for the questions and the interest!