Skip to main content

Reintroducing MegaPortal for both iOS and macOS and ChatGPT supported

· 18 min read
Joe JIANG

This article is written in both Chinese and English.

这篇文章将以中英文混排的方式进行。

Showcases​

Stable Diffusion​

The figure above shows images generated by inputting prompts about Iron Man using different Stable Diffusion models loaded with MegaPortal.

上图为用 MegaPortal 加载不同的 Stable Diffusion 模型，然后输入关于钢铁侠的提示语生成的图片。

ChatGPT Combined with IO​

In the first video, Javascript Block pulled the latest posts and combined with user tags into a text question, which is then fed to ChatPT.

第一个视频通过 Javascript Block 拉取最新的帖子并拼成一个带有用户标签的问题吐给 ChatGPT，让 ChatGPT 来进行推荐。

In the second video, OCR results are used as input and transformed to a text question using Javascript Block, which is then fed to ChatGPT.

第二个视频用 OCR 的结果作为输入，通过 Javascript Block 拼成一个文本问题吐给 ChatGPT 来进行回答。

The the following screenshots of MegaPortal on macOS shows ChatGPT generating a Stable Diffusion prompt.

下面是 MegaPortal 在 macOS 上的截图，可以看到 ChatGPT 生成了一个 Stable Diffusion 的提示语。

The reason for adding the ChatGPT model is that although ChatGPT is very large (LLM) and can not run locally, it can be viewed as a text2text model. As a model loader, this tool supports ChatGPT and is not outside the logical scope of this product.

之所以添加 ChatGPT 模型，因为 ChatGPT 虽然非常大（LLM）和不能在本地跑，但是你可以把它看作是个 text2text 模型，本工具作为一个模型加载器，支持 ChatGPT 并没有脱离本产品逻辑。

Beta testing ChatGPT on MegaPortal​

ChatGPT on MegaPortal is currently in beta testing and intended for limited use, please don't send it sensitive data.

ChatGPT on MegaPortal 目前处于测试阶段，仅供有限使用，请不要发送敏感数据。

You can join the beta testing program by clicking this link:

你可以通过点击这个链接加入测试计划：

https://testflight.apple.com/join/4E61E02T

What is MegaPortal?​

MegaPortal is an easy-to-use AI model loader designed for Apple devices. Here are three key selling points that ChatGPT helped me come up with:

  1. Easy to Use: MegaPortal snippets are composed of basic, easy-to-use visual blocks that can be configured with minimal effort.
  2. AI Accessible to All: MegaPortal is intended for use by not just AI experts and developers, but also non-technical users. It is a user-friendly tool that can be utilized by anyone looking to quickly test, utilize, or share AI models.
  3. Local and Privacy First: All AI Models Run Locally on your device, and all input data is processed locally as well. MegaPortal does not collect any data from you or your device.

For more details: https://docs.getmegaportal.com/

MegaPortal 是一个为苹果设备设计的易用 AI 模型加载工具，下面是我让 ChatGPT 帮我想的三点卖点：

  1. 易于使用：MegaPortal代码段由基本的、易于使用的可视块组成，可以通过低成本进行配置。
  2. AI 适用于所有人：MegaPortal 不仅面向 AI 专家和开发人员，也面向非技术用户。它是一个用户友好型工具，可以被任何希望快速测试、利用或分享 AI 模型的人使用。
  3. 本地隐私优先：所有的 AI 模型都在您的设备上本地运行，所有的输入数据也在本地处理。MegaPortal 不会从您或您的设备中收集任何数据。

详情请访问 MegaPortal 的文档站点：https://docs.getmegaportal.com/ 。同时，MegaPortal 是一个小巧（5 MB, gzipped）且免费的应用 ✌️。

What is it used for?​

MegaPortal is a document-type software, and its executable files are called snippets. Snippets are composed of multiple Blocks, as shown in the following example:

MegaPortal 是一个文档类型软件，执行文件称为 Snippet，Snippet 由若干个 Block 组成，例如下面是一个 Snippet 的例子：

This snippet implements a filter, generating an image output that goes through the AnimeGANv2 model to transform the image into an anime-style image. Since this model outputs images with a size of 512x512, the SRGAN super-resolution model is then applied to upscale the image to a size of 2048x2048. Here are some brief introductions to the relevant models:

这个 Snippet 实现了一个人脸滤镜，以图片作为输出，先经过 AnimeGANv2 将图片变成 Anime 风格的图片，由于这个模型输出的是 512X512 大小的图片，再通过 SRGAN 超解析模型变成 2048X2048 大小的图片。相关模型介绍：

  1. https://github.com/bryandlee/animegan2-pytorch
  2. https://github.com/dongheehand/SRGAN-PyTorch

To further illustrate the usage of MegaPortal, below are three Snippets that I have configured:

为了进一步介绍 MegaPortal 的用途，下面介绍三个本人配置的 Snippet：

  1. The first video demonstrates the effect of the above-mentioned filter, and how it worked has been introduced earlier.

  2. As a Genshin Impact player, the second video introduced a Snippet for recommending Genshin Impact gacha. It accepts the a picture of character list from miHoYo's app as input, processes the text information in the image through MegaPortal's Visual Text Recognition Block, and then processes it through a Javascript Execution Block. This JS Block mainly handles similar text fitting (due to the many rare characters in Genshin Impact, such as "云堇" may be recognized as "云革", which needs to be fitted), recommendation, and display results. By the way, the Javascript Execution Block is written through a VSCode plug-in, which is equipped with type prompts and compilation functions.

  3. The third video demonstrates a Snippet configured to generate images from text using the Stable Diffusion model. The first Block is a Javascript Execution Block, which is used to display the form and pass the input content to the second Block. The second Block loads an SD model sized as at least 2G to generate images. It should be noted that, due to the large size of the SD model, it takes some time to load when it is first run. Also, although the video above demonstrates the results of running on an iPhone, in reality, I have only been able to successfully run a standard SD model, and other models crash during runtime. This is also what I am seeking help with in the "Seeking Help" section.

  4. 第一个视频演示的是上述人脸滤镜的效果，配置原理已经介绍过；

  5. 由于本人是原神玩家，第二个视频配置了一个用于推荐原神抽卡的 Snippet。它接受米游社的角色列表图片作为输入，通过 MegaPortal 的 Visual Text Recognition Block 处理后得到图片中的文字信息。接着，再通过一个 Javascript Execution Block 进行处理。这个 JS Block 主要承担相似文字拟合（由于原神生僻字太多，例如“云堇”可能会被识别成“云革”，需要进行拟合）、推荐和表单展示逻辑功能。顺带一提，Javascript Execution Block 是通过 VSCode 插件进行书写的，插件配备类型提示和编译功能。

  6. 第三个视频则是配置了一个利用 Stable Diffusion 模型来通过文字生成图片的 Snippet。第一个 Block 是 Javascript Execution Block，用来展示表单和将表单输入内容传递给第二个 Block。第二个 Block 则加载一个起步为 2G 的 SD 模型来生成图片。需要注意的是，由于 SD 模型较大，第一次运行需要一定的加载时间。同时，虽然上面视频演示的是 iPhone 的运行结果，但实际上，目前我只成功地跑起了一个标准的 SD 模型，其他模型运行时都会崩溃。这也是我在“寻求帮助”部分想要求助的内容。

In the Snippet Center of the application, you can discover more interesting Snippets, and you can also configure your own Snippets to share the Snippet file or publish it to the Snippet Center for other users to download.

在应用的 Snippet Center 中可以挖掘更多的有趣的 Snippet，你也可以配置自己的 Snippet，通过分享 Snippet 文件或者发布到 Snippet Center 中供其他用户下载。

How to use it​

In this section, we will mainly use configuring a Stable Diffusion Snippet as an example to introduce the usage of MegaPortal.

本环节主要以配置一个 Stable Diffusion 的 Snippet 为例，向大家讲解 MegaPortal 的使用方法。

To use all the features of MegaPortal, especially Stable Diffusion, please upgrade your device to at least iOS/iPadOS 16.2 and macOS 13.1.

为了使用 MegaPortal 的全部功能（特指 Stable Diffusion），请将设备至少升级至 iOS/iPadOS 16.2, macOS 13.1。

Download​

Please visit https://www.getmegaportal.com/ to download the latest version. Currently, both iOS and macOS versions are available for download.

请访问 https://www.getmegaportal.com/ 下载最新版本，目前 iOS 和 macOS 版本都可下载。

Configure an AI image Generation Snippet​

The following will describe the process of configuring a Snippet on macOS.

下面将在 macOS 下为例，讲述配置一个 Snippet 的过程。

Step 1: Open MegaPortal and click "New Document" to create a new file. Then, download a Stable Diffusion Snippet as a template for configuration, and double-click to open the file.

第一步，打开 MegaPortal，点击新建文档，创建一个空文件。然后，下载一个 Stable Diffusion 的 Snippet 作为模板配置，然后双击打开这个文件：

As the model comes with a prepared Stable Diffusion model, the program will automatically start downloading related models when opened, which may take some time. If the network is not good, please close the program and download a model using a download tool in the "Stable Diffusion Model Download" section below.

由于模型中自带 Stable Diffusion 模型，打开后程序会自动开始下载相关模型，模型下载需要一定的时间。如果网络不好，请先关闭程序，在下面「Stable Diffusion 模型下载」部分用下载工具先下载一个模型。

After downloading the model, you can click on the AI Model Application block to configure it. Once configured, click "Save" to save the changes. Since the previous download may still be in progress, you may need to completely close MegaPortal to interrupt the download.

当你下载完模型后，可以点开 AI Model Application 的 Block 进行配置，配置完后点「Save」保存，由于之前的下载可能在进行中，目前只能完全关闭 MegaPortal 才能中断下载。

To run this Snippet, you can reopen the file and click on the "Play" button.

重新打开文件，点「播放」按钮即可运行这个 Snippet：

Right-click on the image and you can save it to the Photos app.

单击右键就可以将图片保存至 Photos 应用。

Clear Cache​

To clear the cache of MegaPortal, you can follow these steps:

  1. Click the "More" button;
  2. Click the "Configuration" button;
  3. Click the "Local Caches" button;
  4. Click "Delete All" or long press/right click on a specific item to delete it.

运行一段时间之后，应用会缓存很多的模型文件，可以通过下面步骤清理缓存：

  1. 点击「More」按钮；
  2. 点击 Configuration 按钮；
  3. 点击「Local Caches」按钮；
  4. 点击「Delete All」或者长按 / 右键某一条目进行删除；

Model Download​

The CoreML format models corresponding to the open-source SD project have been converted and are available for download as needed.

  1. Due to space and bandwidth constraints, most models do not come with a Safety Checker. Please refrain from creating NSFW content 🙏.
  2. The models range in size from 2GB to 4GB and can be downloaded at reasonable speeds using the company VPN.

根据网上的开源 SD 项目，转换好了对应的 CoreML 格式的模型文件，大家可以根据需求下载。

  1. 由于为了节省空间和带宽，大部分模型中没有带有 Safety Checker，请不要产生 NSFW 内容 🙏
  2. 模型大小为 2G~4G，用公司 VPN 下载的话速度尚可。

Ghibli Style​

Links

  1. Downloa: https://model.getmegaportal.com/ghibli-diffusion-v1-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/Ghibli-Diffusion

Anime Style​

本人最喜欢的一类风格

Links

  1. Download: https://model.getmegaportal.com/8528-diffusion2_split-einsum_compiled.zip
  2. Source: https://huggingface.co/852wa/8528-diffusion

Elden Ring Style​

Links

  1. Download: https://model.getmegaportal.com/eldenRing-v3-pruned-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/elden-ring-diffusion

Classic Disney Style​

Links

  1. Download: https://model.getmegaportal.com/classicAnim-v1-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/classic-anim-diffusion

Redshift Style​

Links

  1. Download: https://model.getmegaportal.com/redshift-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/redshift-diffusion

Spideverse Style​

Links

  1. Download: https://model.getmegaportal.com/spiderverse-v1-pruned-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/spider-verse-diffusion

Archer Style​

Links

  1. Download: https://model.getmegaportal.com/archer-v1-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/archer-diffusion

Anime Style​

Links

  1. Download: https://model.getmegaportal.com/arcane-diffusion-v3-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/Ghibli-Diffusion

Original Stable Diffusion 2-1（iPhone tested）​

Links

  1. Download: https://model.getmegaportal.com/coreml-stable-diffusion-2-1-base_split_einsum_compiled.zip
  2. Source:
  3. https://huggingface.co/stabilityai/stable-diffusion-2-1
  4. https://huggingface.co/pcuenq/coreml-stable-diffusion-2-1-base

Midjourney v4​

Links

  1. Download: https://model.getmegaportal.com/mdjrny-v4-einsum_compiled.zip
  2. Source: https://huggingface.co/prompthero/openjourney/tree/main

Nitro Diffusion​

Links

  1. Download: https://model.getmegaportal.com/nitroDiffusion-v1-einsum_compiled.zip
  2. Source: https://huggingface.co/nitrosocke/Nitro-Diffusion

Thanks!​

  1. Thank you, ChatGPT, for helping me with the translation and programming questions.
  2. Thank you, Copilot, for pair programming with me.
  3. I am grateful to various open-source AI model projects in the community. I cannot thank each of them enough 🙏.