Saturday, September 8, 2018

Newton MessagePad iPad Quest

This blog post documents my quest to get Newton OS 2.1 (717006 ROM) to run on a 2018 iPad Pro with Apple Pencil.

Newton MessagePad 2100
I began this quest a week ago. I own three Newton MessagePads and recently purchased an iPad Pro with the Apple Pencil. I had experimented with Einstein, the free Newton emulator, several years ago and had been able to get it to run on my Mac but without a stylus the experience was lacking. Since then the Einstein project had included iOS compatibility, so I was hoping I could get Einstein running on my iPad.

unna.dillernet.com website
First I downloaded the latest Einstein software (v. 2017.1.0) which includes an iOS version from github.com/pguyot/Einstein. After a few hours I was able to get it working on Mac OS X using the debugger images I had downloaded from the United Network of Newton Archives (UNNA) site at www.unna.org or unna.dillernet.com several years ago. While most of the UNNA direct URLs no longer work, the Archive icon on the unna.dillernet.com site still works and allows software to be browsed and downloaded.  There is also a torrent posted there that contains all of the software as of 2006.

Because Einstein cannot be made available from the Apple App store, you must build it onto your iOS device yourself. To do that you need the following:

  • Able to connect iOS device to Mac running Xcode using appropriate cable. For me I connected my iPad Pro with an Apple USB/Lightning cable which comes with the iPad Pro for charging.
  • Current version of the free Apple Xcode software for Mac OS X. This can be downloaded from the App store. I used 9.4.1.
  • An Apple ID set up as a developer account.  To build and install applications for your personal use you just need a free personal developer account. This prevents releasing your apps to the store, but is sufficient for installing an app onto your own iPhone or iPad.  Getting my Apple ID to be set up as a personal developer account required going to the Preferences Account pane in Xcode.

Though successful at getting Einstein to run on my Mac and getting a successful build on the iPad Pro, I was unable to get it to actually run on the iPad Pro. The screen would display and the chime would occur but then it would lock up. After trying a number of things with no success I resolved to dump an official ROM from my used Newton MessagePad 2100 and try to use that rather than the debugger files that I had found on the UNNA site.

First I had to locate my MP 2100 and make sure it still booted up. Instructions for dumping a ROM are available here: https://github.com/pguyot/Einstein/wiki/Dumping-The-Rom

Dumping the ROM requires installing software packages on the Newton. For Newton OS 2.x devices the software of choice is the Newton Connection Utilities 1.0 (released in 1997) which is included on the MP 2100 CD-ROM. The older Newton Connection Kit software is for Newton OS 1.x. Since the Newton Connection Utilities only runs on Mac OS versions prior to Mac OS X, it is necessary to use an older Macintosh capable of running Mac OS 9 or earlier. (Note that the NCU can be downloaded from the unna.dillernet.com Archive from folder /apple/connection_utils/ForMac/NewtonConnectionUtilities but it is compressed with Stuffit and will need to be decompressed to use. Alternatively you can download a WinZipped ISO image of the MP 2100 CD-ROM from folder /apple/software/2100CDROM, which also includes NCU 1.0.)

Having a Compaq WL110 Wireless LAN PC Card for my MP 2100 I thought I could install packages to the Newton from the Macintosh over my wireless network and not require any cables. Unfortunately when inserting the Compaq card in the MP 2100 I received the error "Newton cannot recognize this type of card". Maybe the previous owner had removed necessary software from the Newton before selling it to me.

Older Newtons than the MP 2000 used a serial cable to connect the Newton to the Mac and the Newton Connection Kit software for Mac includes a serial cable. Fortunately I had that so I thought connecting my MP 2100 would not be a significant problem. Unfortunately I ran into two roadblocks with using the cable.  First, with the MP 2000 and MP 2100 Apple eliminated the standard serial port and replaced it with a unique multi-purpose port called the InterConnect port. In order to use this port with a serial cable you need a special adapter. The original owner of my MP 2100 hadn't sent me one of those.  So I ordered one on eBay which cost me $55 and arrived a week later. The second roadblock is that the older Macintosh I planned to use is a PowerMac G3 Blue and White and it also does not have a standard serial port, so connecting with the cable to the G3 B&W wasn't possible anyway.  So I had to dig out another Mac, a venerable all-in-one Power Macintosh 5400/120. This PowerMac has a standard serial port and a CD drive with a built-in tray and a 3.5" floppy disk drive and is running Mac OS 9.1. What it lacks is USB ports, so I had to also locate an ADB keyboard and mouse.  At first the 5400/120 refused to boot, giving me the no boot disk icon with the flashing question mark.  Knowing that sometimes hard drives that haven't been used in a long while can require a few attempts to get them going I powered off the Mac and started it again, and again, and then again.  The fourth time the hard drive finally spun up and I found myself looking successfully at the Mac OS 9.1 desktop.

The next step was to install the Newton Connection Utilities software onto the PowerMac 5400/120 from the MP 2100 CD-ROM. Fortunately the CD-ROM drive in the PM 5400/120 is a tray and so I didn't have to locate a CD-ROM cartridge. The software installed correctly but then required a reboot. Fortunately the hard drive spun up successfully again. At this point I figured that I would be able to connect the MP 2100 to the PowerMac 5400/120 to install packages onto the Newton, so I turned my attention to acquiring what I would need to install on the PowerMac and Newton in order to dump the Newton ROM.

According to the directions on the Einstein site I would need either Newtsbug or Hammer, both of which are debuggers that are included as part of the Lantern DDK. I checked the UNNA site at unna.dillernet.com and clicked on the Archive icon which allows browsing the software that they have to download.  Inside folder /apple/development/DDKs I found the compressed file FullLanternDDK.sit (the sit suffix indicates compressed with the StuffIt compression utility). I was able to download it to my MacMini and decompress it while running Mac OS X.

Now the problem was how to get the application from my MacMini to my PowerMac 5400/120. I could burn the files needed to a CD-ROM using an external USB DVD-ROM drive but I only had blank DVD-ROM discs and the PowerMac 5400/120 only has a CD-ROM drive. Connecting a 3.5" floppy disk drive to the MacMini would also not work and even if it had the 1.44 MB storage would make copying the 50 MB files impossible.  Using a USB flash also won't work because the PowerMac 5400/120 has no USB ports.

I thought that uploading the files to Google drive might work, but the version of Netscape Communicator on the PowerMac 5400/1200 is too old to access any https sites. The PowerMac 5400/120 has an Ethernet port but attempts to access a network drive attached to my router also proved unsuccessful. Stymied at every turn it seemed.

The next technology I tried is an Iomega Zip drive. I have both USB and SCSI versions of the external Zip drives so my thinking was to connect a USB drive to my MacMini, copy the files onto a Zip disk, and then read them off of a SCSI attached Zip drive on the PowerMac 5400/120. Connecting a USB Zip drive to the MacMini worked but for some reason I could not write to a blank Mac OS Standard formatted Zip disk while in Mac OS X 10.13.

I decided to attempt using the G3 B&W which runs both Mac OS X 10.4 and Mac OS 9.2 as a go-between between the MacMini and the PowerMac to transfer the files. First I would copy the files to a USB flash drive, then power up my G3 B&W in Mac OS 9.2 and copy the files from the flash drive to it.  Then connect the USB Zip drive to the G3 B&W and try to write the files to the Zip disk. And finally try to read and copy the files using a SCSI Zip drive on the PowerMac 5400/120.

Copying the files to a flash drive was not a problem but the G3 B&W couldn't read the flash drive in OS 9.2 and it locked up the computer trying.  I rebooted into Mac OS X 10.4.11 and tried again.  This time the flash drive could be read and I was able to copy the files to the documents folder. I then ejected the flash drive and hooked up the USB Zip drive.  I was able to copy the files to the Zip disk, but I had to be selective because the Zip disk only holds 100 MB and I had 148 MB of files I had wanted to copy.  I decided to copy only the Lantern DDK which is about 50 MB. Unfortunately the SCSI Zip disk drive not only would not mount the disk on the desktop but also would not allow me to eject the disk. To manually force eject the disk I had to find a paperclip, straighten one end of it, and insert it into a tiny hole at the back of the drive.  I tried several disks and found that none of them mounted, including the Zip Tools disk.  Fortunately I have two SCSI Zip drives, so I disconnected the drive and connected the second Zip drive (same SCSI cable and power supply though).  This did not solve the mounting problem.  Finally I decided to try the other Zip power supply and moments later the Zip Tools disk appeared successfully on my PowerMac 5400/120 desktop.  I experimented a little more to make sure it wasn't a fluke and that the power supply was really to blame.  It turns out that both drives work fine, but one of the power supplies had somehow gone bad while sitting in its box in my closet.

12 hours after starting this process this morning, I finally had been able to get the Lantern DDK onto a PowerMac that I could use to connect to the MP 2100. The next step would be to see how to connect the MP 2100 using the serial cable and start up Newton Connection Utilities. I connected the serial cable to the InterConnect port adapter and the other end to the modem serial port on the PowerMac and then plugged the adapter into the Newton InterConnect port.  I downloaded and read the instructions for the Newton Connection Utilities and it indicated that you have to go into Preferences in the program and indicate which port you are going to use, so I selected the modem port in Preferences.  Then on the Newton I went to Extras > Dock and initiated a serial connection.  Success.  Next step to install the Debugger Connection package from the Lantern DDK Hammer folder.

I was able to install the package by selecting install package on the Newton. It took a bit for the file list to be sent over from the Mac but then I was able to navigate to the Hammer folder and select the Debugger Connection package. Then I exited Newton Connection Utilities.

I then started the Hammer application on the PowerMac 5400/120 and I was presented with a standard open file dialog. According to the Einstein directions you navigate to the appropriate debugging image based on the Newton you are using.  The directions say "choose the debugging image corresponding to your Newton (Senior CirrusNoDebug image, Senior DCirrusNoDebug image or Newt KNoDebug image for the MP2x00 US, MP2100 or eMate 300 respectively)."  I am using a MP 2100 but unfortunately there was no Senior DCirrusNoDebug image in the folders included with Hammer. I did find though that in the UNNA Archive folder developer/Debugger_Images there is a compressed Stuffit file MP2100_D.sit. This file when unstuffed has a file named "Senior DCirrusNoDebug image" so that is the file I first tried to use. Then once Hammer told me to connect to the Newton device I opened the Extras drawer on the Newton and tapped the Debugger Connection package and selected "connect". Unfortunately Hammer then reported that the ROM didn't match the image. I exited out of Hammer, rebooted the Newton and started Hammer again but this time selected the "Senior CirrusNoDebug image" and after waiting a minute and not getting any error from Hammer decided to soldier on. I went to the File menu and select the "Save Memory..." option and then save 0 to 00800000.  I saved that as "MP2100.rom". The Mac cursor changed to the wristwatch icon and I assumed it was working, so I left the Newton as is and waited and waited and waited some more. Unfortunately the instructions on Einstein only indicate that over a serial connection this takes a long time, but it doesn't give any indication just how long it could be.

After an hour of watching I decided to call it a night and check it in the morning.  While your own mileage may vary it took over 5 hours for the memory save to complete. When it completed the PowerMac cursor returned to the normal pointer.  I exited Hammer and moved my new ROM named MP2100.rom from the PowerMac to a Zip disk and then to a flash drive and then to my MacMini that runs Xcode. The end was in sight.

I copied my new rom file into the ~/Library/Application Support/Einstein Platform folder used by Einstein and changed Einstein's preferences to use the rom. I then started up Einstein and it booted fine, verifying that the new rom was functional.

I started up Xcode and plugged in my iPad via the Lightning cable, which then automatically started up iTunes (which would be needed anyway).  I deleted the existing version of Einstein from my iPad and then in Xcode opened the project, made sure that my iPad was the target and issued the Run command. The application was built successfully on the Newton but couldn't be run because my iPad wasn't set up to trust my developer account.  So on the iPad I went into System Preferences > General > Device Management, selected my developer Apple ID and clicked the "Trust" button.  I also went over to iTunes and with my iPad selected went into File Sharing and selected iOSEinstein.  I then copied my MP2100.rom file to the desktop and renamed that version 717006.rom (which the Einstein instructions say it must be named), and then dragged it to the iTunes iOSEinstein Documents list to add it there.

[Update: Using a free developer account to install the software does work, but the installation will stop working after 7 days, requiring a rebuild. The packages installed should be retained across rebuilds though so this is more annoyance than anything else.]

Then I returned to Xcode and issued the Run command again.  This time the application Einstein started up on the iPad and I heard the familiar Newton chime and then this screen...



Unfortunately that is as far as it went.  This is actually the exact same issue I had a week and $55 ago. I learned a few things along the way but the destination did not turn out to be what I had hoped. If any of you have been successful at getting Einstein to run on an iPad Pro please send me a note.  I'd love to know your story.

To sum up, here are the important lessons I learned along this quest:

  • The emergency force eject hole on Zip drives is in the back, not the front.
  • Unlike earlier Newtons the MP 2000 and 2100 have a custom Apple InterConnect port that requires a small adapter to allow it to be used with standard serial cables. (Thanks, you clever Apple engineers.)
  • An Iomega Zip drive power supply can go bad for no reason other than age. But the drive mechanisms and disks themselves are highly reliable, even after 20+ years.
  • CD-ROM drives cannot read DVD-ROM disks . (Yes, I already knew that, but I tried it anyway.)
  • I have a very hard time getting rid of computer stuff. (Even when I know things are broken.)
12/25/19 Update: SUCCESS! Following directions provided in a comment below I have been able to get it working.  Further explanation of the issue is here: https://github.com/pguyot/Einstein/issues/53

12/27/19 Update: Installation of Newton packages using iTunes couldn't be easier, as it follows the same method as installing the ROM. Just connect the iPad and in iTunes select the device and File Sharing and iOSEinstein, and then just drag any Newton pkg files desired into the iTunes window. So far it seems to be working flawlessly with the exception of the Newton clock which is not handling the 21st century well.