Saturday, September 8, 2018

Newton MessagePad iPad Quest

This blog post documents my quest to get Newton OS 2.1 (717006 ROM) to run on a 2018 iPad Pro with Apple Pencil.

Newton MessagePad 2100
I began this quest a week ago. I own three Newton MessagePads and recently purchased an iPad Pro with the Apple Pencil. I had experimented with Einstein, the free Newton emulator, several years ago and had been able to get it to run on my Mac but without a stylus the experience was lacking. Since then the Einstein project had included iOS compatibility, so I was hoping I could get Einstein running on my iPad.

unna.dillernet.com website
First I downloaded the latest Einstein software (v. 2017.1.0) which includes an iOS version from github.com/pguyot/Einstein. After a few hours I was able to get it working on Mac OS X using the debugger images I had downloaded from the United Network of Newton Archives (UNNA) site at www.unna.org or unna.dillernet.com several years ago. While most of the UNNA direct URLs no longer work, the Archive icon on the unna.dillernet.com site still works and allows software to be browsed and downloaded.  There is also a torrent posted there that contains all of the software as of 2006.

Because Einstein cannot be made available from the Apple App store, you must build it onto your iOS device yourself. To do that you need the following:

  • Able to connect iOS device to Mac running Xcode using appropriate cable. For me I connected my iPad Pro with an Apple USB/Lightning cable which comes with the iPad Pro for charging.
  • Current version of the free Apple Xcode software for Mac OS X. This can be downloaded from the App store. I used 9.4.1.
  • An Apple ID set up as a developer account.  To build and install applications for your personal use you just need a free personal developer account. This prevents releasing your apps to the store, but is sufficient for installing an app onto your own iPhone or iPad.  Getting my Apple ID to be set up as a personal developer account required going to the Preferences Account pane in Xcode.

Though successful at getting Einstein to run on my Mac and getting a successful build on the iPad Pro, I was unable to get it to actually run on the iPad Pro. The screen would display and the chime would occur but then it would lock up. After trying a number of things with no success I resolved to dump an official ROM from my used Newton MessagePad 2100 and try to use that rather than the debugger files that I had found on the UNNA site.

First I had to locate my MP 2100 and make sure it still booted up. Instructions for dumping a ROM are available here: https://github.com/pguyot/Einstein/wiki/Dumping-The-Rom

Dumping the ROM requires installing software packages on the Newton. For Newton OS 2.x devices the software of choice is the Newton Connection Utilities 1.0 (released in 1997) which is included on the MP 2100 CD-ROM. The older Newton Connection Kit software is for Newton OS 1.x. Since the Newton Connection Utilities only runs on Mac OS versions prior to Mac OS X, it is necessary to use an older Macintosh capable of running Mac OS 9 or earlier. (Note that the NCU can be downloaded from the unna.dillernet.com Archive from folder /apple/connection_utils/ForMac/NewtonConnectionUtilities but it is compressed with Stuffit and will need to be decompressed to use. Alternatively you can download a WinZipped ISO image of the MP 2100 CD-ROM from folder /apple/software/2100CDROM, which also includes NCU 1.0.)

Having a Compaq WL110 Wireless LAN PC Card for my MP 2100 I thought I could install packages to the Newton from the Macintosh over my wireless network and not require any cables. Unfortunately when inserting the Compaq card in the MP 2100 I received the error "Newton cannot recognize this type of card". Maybe the previous owner had removed necessary software from the Newton before selling it to me.

Older Newtons than the MP 2000 used a serial cable to connect the Newton to the Mac and the Newton Connection Kit software for Mac includes a serial cable. Fortunately I had that so I thought connecting my MP 2100 would not be a significant problem. Unfortunately I ran into two roadblocks with using the cable.  First, with the MP 2000 and MP 2100 Apple eliminated the standard serial port and replaced it with a unique multi-purpose port called the InterConnect port. In order to use this port with a serial cable you need a special adapter. The original owner of my MP 2100 hadn't sent me one of those.  So I ordered one on eBay which cost me $55 and arrived a week later. The second roadblock is that the older Macintosh I planned to use is a PowerMac G3 Blue and White and it also does not have a standard serial port, so connecting with the cable to the G3 B&W wasn't possible anyway.  So I had to dig out another Mac, a venerable all-in-one Power Macintosh 5400/120. This PowerMac has a standard serial port and a CD drive with a built-in tray and a 3.5" floppy disk drive and is running Mac OS 9.1. What it lacks is USB ports, so I had to also locate an ADB keyboard and mouse.  At first the 5400/120 refused to boot, giving me the no boot disk icon with the flashing question mark.  Knowing that sometimes hard drives that haven't been used in a long while can require a few attempts to get them going I powered off the Mac and started it again, and again, and then again.  The fourth time the hard drive finally spun up and I found myself looking successfully at the Mac OS 9.1 desktop.

The next step was to install the Newton Connection Utilities software onto the PowerMac 5400/120 from the MP 2100 CD-ROM. Fortunately the CD-ROM drive in the PM 5400/120 is a tray and so I didn't have to locate a CD-ROM cartridge. The software installed correctly but then required a reboot. Fortunately the hard drive spun up successfully again. At this point I figured that I would be able to connect the MP 2100 to the PowerMac 5400/120 to install packages onto the Newton, so I turned my attention to acquiring what I would need to install on the PowerMac and Newton in order to dump the Newton ROM.

According to the directions on the Einstein site I would need either Newtsbug or Hammer, both of which are debuggers that are included as part of the Lantern DDK. I checked the UNNA site at unna.dillernet.com and clicked on the Archive icon which allows browsing the software that they have to download.  Inside folder /apple/development/DDKs I found the compressed file FullLanternDDK.sit (the sit suffix indicates compressed with the StuffIt compression utility). I was able to download it to my MacMini and decompress it while running Mac OS X.

Now the problem was how to get the application from my MacMini to my PowerMac 5400/120. I could burn the files needed to a CD-ROM using an external USB DVD-ROM drive but I only had blank DVD-ROM discs and the PowerMac 5400/120 only has a CD-ROM drive. Connecting a 3.5" floppy disk drive to the MacMini would also not work and even if it had the 1.44 MB storage would make copying the 50 MB files impossible.  Using a USB flash also won't work because the PowerMac 5400/120 has no USB ports.

I thought that uploading the files to Google drive might work, but the version of Netscape Communicator on the PowerMac 5400/1200 is too old to access any https sites. The PowerMac 5400/120 has an Ethernet port but attempts to access a network drive attached to my router also proved unsuccessful. Stymied at every turn it seemed.

The next technology I tried is an Iomega Zip drive. I have both USB and SCSI versions of the external Zip drives so my thinking was to connect a USB drive to my MacMini, copy the files onto a Zip disk, and then read them off of a SCSI attached Zip drive on the PowerMac 5400/120. Connecting a USB Zip drive to the MacMini worked but for some reason I could not write to a blank Mac OS Standard formatted Zip disk while in Mac OS X 10.13.

I decided to attempt using the G3 B&W which runs both Mac OS X 10.4 and Mac OS 9.2 as a go-between between the MacMini and the PowerMac to transfer the files. First I would copy the files to a USB flash drive, then power up my G3 B&W in Mac OS 9.2 and copy the files from the flash drive to it.  Then connect the USB Zip drive to the G3 B&W and try to write the files to the Zip disk. And finally try to read and copy the files using a SCSI Zip drive on the PowerMac 5400/120.

Copying the files to a flash drive was not a problem but the G3 B&W couldn't read the flash drive in OS 9.2 and it locked up the computer trying.  I rebooted into Mac OS X 10.4.11 and tried again.  This time the flash drive could be read and I was able to copy the files to the documents folder. I then ejected the flash drive and hooked up the USB Zip drive.  I was able to copy the files to the Zip disk, but I had to be selective because the Zip disk only holds 100 MB and I had 148 MB of files I had wanted to copy.  I decided to copy only the Lantern DDK which is about 50 MB. Unfortunately the SCSI Zip disk drive not only would not mount the disk on the desktop but also would not allow me to eject the disk. To manually force eject the disk I had to find a paperclip, straighten one end of it, and insert it into a tiny hole at the back of the drive.  I tried several disks and found that none of them mounted, including the Zip Tools disk.  Fortunately I have two SCSI Zip drives, so I disconnected the drive and connected the second Zip drive (same SCSI cable and power supply though).  This did not solve the mounting problem.  Finally I decided to try the other Zip power supply and moments later the Zip Tools disk appeared successfully on my PowerMac 5400/120 desktop.  I experimented a little more to make sure it wasn't a fluke and that the power supply was really to blame.  It turns out that both drives work fine, but one of the power supplies had somehow gone bad while sitting in its box in my closet.

12 hours after starting this process this morning, I finally had been able to get the Lantern DDK onto a PowerMac that I could use to connect to the MP 2100. The next step would be to see how to connect the MP 2100 using the serial cable and start up Newton Connection Utilities. I connected the serial cable to the InterConnect port adapter and the other end to the modem serial port on the PowerMac and then plugged the adapter into the Newton InterConnect port.  I downloaded and read the instructions for the Newton Connection Utilities and it indicated that you have to go into Preferences in the program and indicate which port you are going to use, so I selected the modem port in Preferences.  Then on the Newton I went to Extras > Dock and initiated a serial connection.  Success.  Next step to install the Debugger Connection package from the Lantern DDK Hammer folder.

I was able to install the package by selecting install package on the Newton. It took a bit for the file list to be sent over from the Mac but then I was able to navigate to the Hammer folder and select the Debugger Connection package. Then I exited Newton Connection Utilities.

I then started the Hammer application on the PowerMac 5400/120 and I was presented with a standard open file dialog. According to the Einstein directions you navigate to the appropriate debugging image based on the Newton you are using.  The directions say "choose the debugging image corresponding to your Newton (Senior CirrusNoDebug image, Senior DCirrusNoDebug image or Newt KNoDebug image for the MP2x00 US, MP2100 or eMate 300 respectively)."  I am using a MP 2100 but unfortunately there was no Senior DCirrusNoDebug image in the folders included with Hammer. I did find though that in the UNNA Archive folder developer/Debugger_Images there is a compressed Stuffit file MP2100_D.sit. This file when unstuffed has a file named "Senior DCirrusNoDebug image" so that is the file I first tried to use. Then once Hammer told me to connect to the Newton device I opened the Extras drawer on the Newton and tapped the Debugger Connection package and selected "connect". Unfortunately Hammer then reported that the ROM didn't match the image. I exited out of Hammer, rebooted the Newton and started Hammer again but this time selected the "Senior CirrusNoDebug image" and after waiting a minute and not getting any error from Hammer decided to soldier on. I went to the File menu and select the "Save Memory..." option and then save 0 to 00800000.  I saved that as "MP2100.rom". The Mac cursor changed to the wristwatch icon and I assumed it was working, so I left the Newton as is and waited and waited and waited some more. Unfortunately the instructions on Einstein only indicate that over a serial connection this takes a long time, but it doesn't give any indication just how long it could be.

After an hour of watching I decided to call it a night and check it in the morning.  While your own mileage may vary it took over 5 hours for the memory save to complete. When it completed the PowerMac cursor returned to the normal pointer.  I exited Hammer and moved my new ROM named MP2100.rom from the PowerMac to a Zip disk and then to a flash drive and then to my MacMini that runs Xcode. The end was in sight.

I copied my new rom file into the ~/Library/Application Support/Einstein Platform folder used by Einstein and changed Einstein's preferences to use the rom. I then started up Einstein and it booted fine, verifying that the new rom was functional.

I started up Xcode and plugged in my iPad via the Lightning cable, which then automatically started up iTunes (which would be needed anyway).  I deleted the existing version of Einstein from my iPad and then in Xcode opened the project, made sure that my iPad was the target and issued the Run command. The application was built successfully on the Newton but couldn't be run because my iPad wasn't set up to trust my developer account.  So on the iPad I went into System Preferences > General > Device Management, selected my developer Apple ID and clicked the "Trust" button.  I also went over to iTunes and with my iPad selected went into File Sharing and selected iOSEinstein.  I then copied my MP2100.rom file to the desktop and renamed that version 717006.rom (which the Einstein instructions say it must be named), and then dragged it to the iTunes iOSEinstein Documents list to add it there.

[Update: Using a free developer account to install the software does work, but the installation will stop working after 7 days, requiring a rebuild. The packages installed should be retained across rebuilds though so this is more annoyance than anything else.]

Then I returned to Xcode and issued the Run command again.  This time the application Einstein started up on the iPad and I heard the familiar Newton chime and then this screen...



Unfortunately that is as far as it went.  This is actually the exact same issue I had a week and $55 ago. I learned a few things along the way but the destination did not turn out to be what I had hoped. If any of you have been successful at getting Einstein to run on an iPad Pro please send me a note.  I'd love to know your story.

To sum up, here are the important lessons I learned along this quest:

  • The emergency force eject hole on Zip drives is in the back, not the front.
  • Unlike earlier Newtons the MP 2000 and 2100 have a custom Apple InterConnect port that requires a small adapter to allow it to be used with standard serial cables. (Thanks, you clever Apple engineers.)
  • An Iomega Zip drive power supply can go bad for no reason other than age. But the drive mechanisms and disks themselves are highly reliable, even after 20+ years.
  • CD-ROM drives cannot read DVD-ROM disks . (Yes, I already knew that, but I tried it anyway.)
  • I have a very hard time getting rid of computer stuff. (Even when I know things are broken.)
12/25/19 Update: SUCCESS! Following directions provided in a comment below I have been able to get it working.  Further explanation of the issue is here: https://github.com/pguyot/Einstein/issues/53

12/27/19 Update: Installation of Newton packages using iTunes couldn't be easier, as it follows the same method as installing the ROM. Just connect the iPad and in iTunes select the device and File Sharing and iOSEinstein, and then just drag any Newton pkg files desired into the iTunes window. So far it seems to be working flawlessly with the exception of the Newton clock which is not handling the 21st century well.


9 comments:

  1. One update to this is that I also found that sometimes Newton packages will not have the correct file creator and type codes to be recognized by the Newton Connection Utilities. A solution to this is the freeware program FileType which runs on Mac OS 8 and 9. It can be used to assign pkgx and pkg to creator code and type respectively.

    ReplyDelete
  2. I had the same problem until I found the solution on the Issues section of GitHub:

    If nothing else, I can confirm that simply commenting out the ROM patch in line 332 of Emulator/Serial/TBasicSerialPortManager.cp eliminates the problem and makes the system boot normally again:

    TJITGenericPatchFindAndReplace gEnableSerialPort(0x00800634, 0x00900000,
    (KUInt32[]){2, 'extr', 0},
    (KUInt32[]){1, '~xtr'},
    "Enable 'extr' serial port.\n");

    ReplyDelete
    Replies
    1. Thank you for the suggestion. Since it has been a year since I failed at this before I had to do a few things more. I updated my iPad Pro (10.5-inch) to iOS 13.3 and XCode to 11.3. In XCode I searched my project for the text TJITGenericPatchFindAndReplace and located the file TBasicSerialPortManager.cp. I then selected that and scrolled down to line 132 and commented out that set of lines with // and saved.

      I targeted my iPad Pro and issued Run to rebuild and execute after that but all I got on the iPad was a black screen instead of the Newton screen. Turns out several of the things I set up a year ago had to be done again.
      - In XCode Preferences go to Accounts and login again with my Developer ID.
      - On the iPad go to General > Device Management > and Trust my Apple Developer account.
      - In iTunes select my iPad and then File Sharing and then iOSEinstein and copy the 717006.rom ROM file over.

      After all this I was able to run from XCode and it started up on my iPad and proceeded past the startup chime. Unfortunately my Apple pen battery is dead so I need to charge it up and then see how it works as a stylus with iOSEinstein.

      Delete
  3. I have been successful but there have been a few things I have noted while playing with iOSEinstein.

    It is nice that both the Apple Pencil and the Apple keyboard work with the emulator but there are some features that will not work:
    - Beaming does not work due to lack of IR hardware
    - Note recordings do not work
    - I have found that leaving the Newton running in the background can cause it to crash when you try to bring it to the foreground. The only solution is to rebuild the app from XCode. (Incidentally the IIGS emulator ActiveGS seems to have the same problem.)
    - The Newton OS 2.0 clock cannot handle time past 6:48:31 pm Jan 5, 2010, so to get the correct month and day and day of week you need to set the year back 28 years. So for 2020, set the year to 1992. Unfortunately because of the issue in the bullet above, you have to set the clock again each time you start.
    - the volume slider has no actual effect on the Newton volume

    ReplyDelete
  4. You're almost there. Install the 711000 patch and the dates will be all good. You can get the patch here:

    http://sourceforge.net/projects/n40hz/files/Patches/Patch-711000.zip/download

    Info about the patch is available on the 40Hz website:

    https://40hz.org/Pages/newton/patches/patch711000/

    ReplyDelete
    Replies
    1. Thank you. I was able to install patch.pkg using iTunes. I notice that it did wipe out my Newton notes though.

      Delete
    2. Unfortunately after installing the patch I have had no luck installing any packages through iTunes. Dragging the packages to the iTunes file sharing window for iOSEinstein does not cause the iOSEinstein to actually install the packages and as far as I know there is nothing in NewtonOS or iTunes to force it. I'm not sure this is related to the 711000 patch or not, but I did not have this problem with packages before. At this point I have a Newton that knows the date and time but I can't really do anything with it except use the built-in apps, which while interesting are lacking functionality that exists in the iPad native apps.

      Delete
  5. That's very strange. After dragging the packages to iTunes, do you Quit iOSEinstein from the "i" icon in iOSEinstein?

    ReplyDelete
  6. Yes though that always locks up iOSEinstein and I then force quit it.

    ReplyDelete