MegaPortal 2.1 is now available for macOS and iOS, introducing a new feature: Stable Diffusion for image-to-image transformation. Additionally, it includes an image editing tool using PencilKit, as well as bug fixes and improvements.
To use the new feature, you need to have to upgrade MegaPortal to version 2.1, Please update your app from the App Store or download the latest version from the MegaPortal website.
The reason for adding the ChatGPT model is that although ChatGPT is very large (LLM) and can not run locally, it can be viewed as a text2text model. As a model loader, this tool supports ChatGPT and is not outside the logical scope of this product.
MegaPortal is an easy-to-use AI model loader designed for Apple devices. Here are three key selling points that ChatGPT helped me come up with:
Easy to Use: MegaPortal snippets are composed of basic, easy-to-use visual blocks that can be configured with minimal effort.
AI Accessible to All: MegaPortal is intended for use by not just AI experts and developers, but also non-technical users. It is a user-friendly tool that can be utilized by anyone looking to quickly test, utilize, or share AI models.
Local and Privacy First: All AI Models Run Locally on your device, and all input data is processed locally as well. MegaPortal does not collect any data from you or your device.
MegaPortal is a document-type software, and its executable files are called snippets. Snippets are composed of multiple Blocks, as shown in the following example:
This snippet implements a filter, generating an image output that goes through the AnimeGANv2 model to transform the image into an anime-style image. Since this model outputs images with a size of 512x512, the SRGAN super-resolution model is then applied to upscale the image to a size of 2048x2048. Here are some brief introductions to the relevant models:
To further illustrate the usage of MegaPortal, below are three Snippets that I have configured:
为了进一步介绍 MegaPortal 的用途，下面介绍三个本人配置的 Snippet：
The first video demonstrates the effect of the above-mentioned filter, and how it worked has been introduced earlier.
As a Genshin Impact player, the second video introduced a Snippet for recommending Genshin Impact gacha. It accepts the a picture of character list from miHoYo's app as input, processes the text information in the image through MegaPortal's Visual Text Recognition Block, and then processes it through a Javascript Execution Block. This JS Block mainly handles similar text fitting (due to the many rare characters in Genshin Impact, such as "云堇" may be recognized as "云革", which needs to be fitted), recommendation, and display results. By the way, the Javascript Execution Block is written through a VSCode plug-in, which is equipped with type prompts and compilation functions.
The third video demonstrates a Snippet configured to generate images from text using the Stable Diffusion model. The first Block is a Javascript Execution Block, which is used to display the form and pass the input content to the second Block. The second Block loads an SD model sized as at least 2G to generate images. It should be noted that, due to the large size of the SD model, it takes some time to load when it is first run. Also, although the video above demonstrates the results of running on an iPhone, in reality, I have only been able to successfully run a standard SD model, and other models crash during runtime. This is also what I am seeking help with in the "Seeking Help" section.
In the Snippet Center of the application, you can discover more interesting Snippets, and you can also configure your own Snippets to share the Snippet file or publish it to the Snippet Center for other users to download.
在应用的 Snippet Center 中可以挖掘更多的有趣的 Snippet，你也可以配置自己的 Snippet，通过分享 Snippet 文件或者发布到 Snippet Center 中供其他用户下载。
The following will describe the process of configuring a Snippet on macOS.
下面将在 macOS 下为例，讲述配置一个 Snippet 的过程。
Step 1: Open MegaPortal and click "New Document" to create a new file. Then, download a Stable Diffusion Snippet as a template for configuration, and double-click to open the file.
As the model comes with a prepared Stable Diffusion model, the program will automatically start downloading related models when opened, which may take some time. If the network is not good, please close the program and download a model using a download tool in the "Stable Diffusion Model Download" section below.
After downloading the model, you can click on the AI Model Application block to configure it. Once configured, click "Save" to save the changes. Since the previous download may still be in progress, you may need to completely close MegaPortal to interrupt the download.
当你下载完模型后，可以点开 AI Model Application 的 Block 进行配置，配置完后点「Save」保存，由于之前的下载可能在进行中，目前只能完全关闭 MegaPortal 才能中断下载。
To run this Snippet, you can reopen the file and click on the "Play" button.
重新打开文件，点「播放」按钮即可运行这个 Snippet：
Right-click on the image and you can save it to the Photos app.
We are excited to share with you MegaPortal for macOS, a small(3.7MB, gzipped) and powerful application designed to run your trained models on your Apple device.
You can easily download it from HERE and take all your trained models with you wherever you go on your macOS and customize the workflow to fit your specific needs.
At this time, MegaPortal supports CoreML models for image-to-image transformations and object detection. It also offers visual text recognition using built-in APIs. We are committed to adding more model types in the future. To test these models, you can easily explore snippets that you can download from the Snippet Center. To learn more about MegaPortal, be sure to check out our tutorial.
We built MegaPortal on top of Mac Catalyst, which also allows us to have a version of iOS submitted to the App Store. We are currently waiting for the review process, but we cannot guarantee if it will be approved or not and it may take longer than expected. With this version, you will be able to run your models on your iPhone and iPad and make your models more portable than ever before.
There are more features for your to explore in MegaPortal. For example, combine text recognition and Javascript block, you can do some interesting things like Genshin gacha recommendation(I am a fan of Genshin Impact 😊):
I hope you will enjoy using MegaPortal and we are looking forward to your feedback. If you have any questions, please feel free to reach out to us at [email protected] or [email protected]
