I think Doom RPG has its own PRNG (in DoomRPG.c in the published code). I've noticed that Entity.c:Entity_aiGoal_MOVE() is buggy. AFAICT, it is intended to move monster towards its target, picking a random direction if two ones would place the monster equally close to the target.
closestPathDist = 9999;
/* [irrelevant code setting i10 omitted] */
if (Entity_checkLineOfSight(entity, srcX, srcY, srcX + 64, srcY, i10)) { /* AFAIU checks if monster can move there */
pathDist = Entity_calcPath(entity, sX + 1, sY, dX, dY, 0, i10);
if (pathDist < closestPathDist) { /* always true! */
closestPathDist = pathDist;
entity->visitOrder[entity->visitOrderCount++] = 2;
}
else if (pathDist == closestPathDist) {
entity->visitOrder[entity->visitOrderCount++] = 2;
}
}
Entity_calcPath() returns some kind of pseudo-distance, which is smaller than 9999.
AFAIU, this means that the east direction, if the monster can move there, is always added to the entity->visitOrder[] array. Then one direction is picked out of entity->visitOrder[] array pseudo-randomly:
switch (entity->visitOrder[(DoomRPG_randNextByte(&entity->doomRpg->random) & 3) % entity->visitOrderCount]) {
/* [code] */
case 2:
entity->monster->x = srcX + 64;
entity->monster->y = srcY;
break;
/* [more stuff] */
AFAIU, the monster moving east happens with probability of about 1/2 - even if it would move monster away from you. Which seems to actually happen.