Maybe I haven't stated my situation clear enough. Motherboard has two ground planes AGND and DGND. They are connected at some point on motherboard. Now, if I use one GND plane on my dayghter board, I will create a ground loop. Right ?
Yes, this creates a ground loop. Most often this doesn't do any harm to the system and still is the better choice than splitting the plane, but it depends on your particular case.
Having a single point star GND between analog and digital creates large amounts of noise if the circuitry is wide spread and has more then one point where signals cross the border between analog and digital. You'd have to route all traces that cross the border along that one connection point, or place all A/D D/A chips over that one connection point. The textbook example works for exactly one chip that crosses that boundary and the AGND/DGND connection is under the chip. If there's more than one chip involved, having AGND/DGND connected under each chip creates ground loops, but still is the better choice than having it connected under one chip. Best choice is to have a solid GND plane. So if you can't change the motherboard, improve its GND plane by making your board with a GND plane.
If can't avoid splitting the GND, connect all A/D and D/A chips (no matter if these are labelled analog or digital) to your analog GND. This reduces noise inside the chip. Now you have a lot of nasty digital signals crossing the AGND/DGND border, this makes up nice loop antennas. You can try to mitigate this issue by making the digital signals high impedance and slow - beware, now they work as receiving antennas for external EMI. So you'll have to additionally filter the signals. If you do this right, it may work. Or isolate them by using suitable isolators. If you connect the chip's GND pins to AGND/DGND as they're labelled, you've got the same issue, but within the chip and you can't do anything about it.
Having a solid GND plane solves this issue and maybe forms a GND loop with your mainboard. This may increase EMI issues on the mainboard, this depends on the circuitry and layout. Worst case is that you'd have to isolate your analog and digital parts to get it right, best case would be that the mainboard circuitry isn't harmed by your GND plane.