Well, I've been banging away at this and cant produce the results you mentioned. I have a file named "hello.htm" and the contents are as follows:
<html>
<head>
<title>PHP Test</title>
</head>
<body>
<?php echo '<p>Hello</p>'; ?>
</body>
</html>
And all I get is a syntax error popup in IE when I view the page, but if i give that same file a php extension it works fine. Is there something I'm doing wrong here?