gh-153319: Read turtledemo scripts with the source file's own encoding

[tonghuaroot]

turtledemo's example viewer opened each demo script for display with
open(self.module.__file__, 'r') (no encoding), decoding with the locale
default encoding. That emits an EncodingWarning under
-X warn_default_encoding and, on a non-UTF-8 locale, can mis-decode a UTF-8
or coding-cookie'd demo, showing mojibake.

This reads the source with tokenize.open() instead, which decodes using the
encoding detected from the file (its PEP 263 coding cookie, else UTF-8). That
is the correct encoding for reading Python source, and it matches how the
module was imported one line above. It is the only unspecified-encoding
open() in Lib/turtledemo (grep-verified).

The viewer is Tkinter GUI code with no test harness (test_turtle.py covers
the turtle module, not the demo runner), so this was verified manually in
both directions: under -X warn_default_encoding the old open() raises an
EncodingWarning and the new tokenize.open() does not, while reading the
file unchanged.

• Issue: turtledemo example viewer decodes scripts with the locale default encoding #153319

[serhiy-storchaka]

LGTM. But did you consider using inspect.getsource()?

[terryjreedy]

AFAIK, turtledemo examples that are not all ascii (are their any?) are utf-8 encoded. Why not just add this to the open call?

[tonghuaroot]

Good point, done. The demo scripts are UTF-8, so I switched to open(..., encoding='utf-8') and dropped the tokenize import.