Modelers Magic Music Community

What is the Modelers Magic Music community?

The Modelers Magic Music community is a neutral, non-profit community aimed at artificial intelligence developers and learners; it offers services for hosting, displaying, downloading, and collaborating on models, datasets, testing environments, and development tools. The community is governed by a council, and its goal is to help build a domestic AI ecosystem through the efforts of various participants in this industry chain.

It is not just a site for downloading model files; it also integrates model cards, dataset descriptions, application demonstrations, tools for collaborative work, Git repositories, access tokens, the openMind development kit, as well as connections for use with the Ascend NPU. Users can find publicly available weights, and they can also create their own repositories to publish models, data, and applications.

Model library

The model library covers tasks such as natural language processing, computer vision, speech, multimodal processing, and generative AI. Model cards display information such as the owner, task type, framework, tags, file details, version, and usage instructions; they may also be linked to application platforms where these models can be tried out online.

Before downloading, it is necessary to read the README file, the model license, as well as the instructions and limitations related to the training data. The fact that a model is hosted on a community platform does not mean that the platform endorses its accuracy, security, or suitability for commercial use. For a given model, there may be original versions, quantized versions, PyTorch versions, MindSpore versions, as well as versions optimized for Ascend.

Methods for downloading the model

For public models, files can be downloaded one by one via a web page, or the entire repository can be cloned using Git and Git LFS. The openMind Hub Client offers single-file downloads, full-repository download via snapshots, caching, version branching, and file filtering, making it suitable for reliably obtaining weights within scripts.

OpenMind Library also allows models to be loaded using the “username/repository name”, with inference carried out through Pipeline, AutoModel, and AutoTokenizer. When trust_remote_code is used, the custom code provided by the repository is executed; it is therefore necessary to review the source code first and test it in an isolated environment.

Model upload and contributions

After registration, it is possible to create personal or organizational model repositories. You can upload files to the upload_folder, which supports Git, Git LFS, and openMind Hub Client, as well as use push_to_hub for the openMind Library trainer.

The repository will automatically create a README model card; contributors should add tasks, data, metrics, environment details, limitations, and licenses.

Access tokens come in Read and Write versions – the Write version allows uploading and downloading data, while the token’s full content is only visible at the time of creation. Separate tokens should be created for automated processes, and they should not be stored in code repositories;

It should be deleted immediately and recreated after being leaked.

Dataset

A dataset repository is used to store, discover, and manage versions of training or evaluation data. Users can filter by task and domain, view files and instructions, and then download them to their local training environment using community tools.

The fact that data is made public does not mean it can be used freely. Before using such data, it is necessary to check the license, personal information, copyright issues, authorization for collection, and any geographical restrictions, especially in cases involving medical data, financial information, facial recognition data, voice recordings, and data obtained from the Internet.

A data card that is missing should not be considered commercially viable by default.

Experience Space

The experience space is used to display online applications built based on models; common forms include ChatBots, images, voice inputs, Notebooks, or custom applications. The model details page can be linked to an experience space, allowing visitors to test inputs and outputs without having to set up a local environment first.

The experience spaces are maintained by different providers, and their startup speed, uptime, computing power, data retention policies, and privacy rules may vary. Do not enter confidential client information, sensitive personal data, or unpublished documents during public demonstrations.

Application and Notebook

Developers can create application-based environments to demonstrate model functionality, or they can use the Notebook environment for experimentation. The official guidance materials show how to load, fine-tune, and run models using NPU resources, making them useful for developers who do not have a local Ascend environment to test compatibility.

Online Notebooks still require management of dependencies, storage, runtime, and resource release. Before starting long-term training, it is necessary to determine the session timeout, disk persistence, quotas, and rules for task interruption, and to export the code, logs, and checkpoints in a timely manner.

openMind Library

The openMind Library is a companion suite for deep learning development; it provides unified APIs for model pre-training, fine-tuning, and inference. It encapsulates the AutoClass, Pipeline, and Trainer interfaces commonly used in Transformers and MindFormers, is compatible with PyTorch and MindSpore, and also offers support for Ascend NPU.

For training, tools such as MindSpeed, Accelerate, PEFT, DeepSpeed, and LLaMA Factory can be used. The actual compatibility depends on openMind, the deep learning framework, CANN, drivers, firmware, and the version of the model implementation; therefore, it is necessary to create an isolated virtual environment in accordance with the documentation before installation.

openMind Hub Client

The openMind Hub Client is a Python and command-line tool for interacting with community repositories; it allows for creating repositories, querying metadata, uploading files, downloading individual files or snapshots, as well as managing local caches. It is suitable for automating training pipelines and preparing environments for offline use.

The client maintains a allowlist and blocklist for download paths to prevent writing to sensitive system directories. Linux allows temporary directories and user directories by default, while Windows allows user directories and certain data drives by default.

When a custom allowlist is defined, the blocklist still has higher priority.

Local reasoning

The quick-start documentation provides examples for Python 3.8, 3.9, and 3.10, and it requires that frameworks such as PyTorch or MindSpore be installed according to the specific tasks. Tasks such as text classification, generation, and visual processing can be carried out using pipelines; after the model weights are downloaded, they rely primarily on the user’s own CPU, GPU, or NPU resources.

Large model files can range from dozens to hundreds of GB in size; before using them locally, it is necessary to check the disk space, memory, video memory, precision levels, quantization methods, and device compatibility. Successful download does not mean that the current hardware can run such files directly.

Pre-training and fine-tuning

The openMind Library provides interfaces such as Trainer and TrainingArguments, which enable further pre-training using models from the community, full fine-tuning, or parameter-efficient fine-tuning via PEFT. Once the process is complete, it is possible to upload the configurations, Tokenizers, and weights back to the community using push_to_hub.

The quality of fine-tuning depends on the data, templates, learning rate, evaluation methods, and the licensing of the base model. Training logs and a decrease in loss alone do not suffice to prove effectiveness; independent validation sets, manual evaluations, and security tests are necessary, along with documentation of the framework, random seed values, and hardware environment.

NPU model availability testing

The model owner or any organization member with write permissions can submit a model availability test to verify whether custom inference scenarios can be executed in an NPU environment. The test script uses inference.py located in the examples directory as its entry point, and it may be accompanied by a requirements file.

If the test is successful, links for downloading the logs and the image will be provided; changes to the repository’s commits, the conversion of it to a private repository, or the removal of a model will result in the termination of the relevant tests. A successful usability test merely indicates that the specified script and environment can run, but it does not mean that accuracy, performance, and production stability have been verified.

Organization and permissions

The organization can centrally manage models, datasets, and experience spaces, and define three types of roles: administrative, editorial, and read-only. Administrators can also manage members, organizational details, and access tokens.

Editors can modify the organization’s repository; read-only members can only access the resources to which they have been granted access.

Once a organization name is created, it cannot be changed. Projects must be removed first before the organization can be deleted; this is a high-risk operation.

Teams should avoid sharing individual Write Tokens, and they should establish review, branching, and backup processes for important models.

Warehouse version management

Both the models and data warehouses make use of Git-style version control, with large files being handled via Git LFS. Users can download specific versions at different revisions, which helps to prevent the inability to reproduce training results after updates are made upstream.

The production build should rely on a specific Commit or Tag, with the hashes of key files being verified. Writing only the name of the main branch can lead to changes as the repository is updated, and it may result in the download of replaced versions or custom code.

Prices and fees

The MoLe community is positioned as a community dedicated to charitable purposes; its models, datasets, documents, and access interfaces can be viewed freely. The official website does not currently specify a unified price for membership subscriptions, nor is there any fixed list of prices for GPUs or NPs that can be used in all the available services.

Free browsing does not mean that all costs are zero. Local downloading incurs storage and network costs, while local training and inference require the user to bear the expenses related to hardware, electricity, or cloud resources.

The resources, quotas, and activity privileges in the online space may vary depending on the specific page.

If the page does not specify the price, one should not rely on articles from third parties to estimate the cost of the package.

Model licenses and commercial use

The models on community platforms, the openMind tools, and in repositories are distinct legal entities. Each model and dataset can have its own license: some permit commercial use, while others require applications, attribution, restrictions on the number of users, or prohibit certain uses.

Before use, it is necessary to check the permissions regarding weights, code, datasets, and the output content. Even if the openMind tool uses a permissive license, this does not automatically change the license of the downloaded models.

GitHub, Gitee, and the open-source status

The main open-source code for openMind Library and openMind Hub Client can be found in the Gitee repositories related to Ascend; the Hub Client is licensed under the Mulan Permissive License Version 2. Modelers also has a feedback repository on Gitee, which is used for communicating about community issues and requirements.

Community web services, accounts, hosted storage, moderation functions, and all the backend code have not been made publicly available as a result of the open-source nature of the toolchain. Therefore, it should be stated that \"the community platform provides an open-source toolchain; the web services themselves do not represent a fully open-source solution.\"

Domestic computing power and framework ecosystems

The MoLe community places emphasis on the optimization of NPU, MindSpore, and the adaptation of domestic models, while also being compatible with PyTorch workflows. Developers can access models that have been optimized for NPU or verified, which reduces the need for trial and error during the transition from traditional GPU ecosystems.

The domestically developed versions still need to be adapted to specific chips, CANN versions, and framework versions. The “Ascend” label on model cards does not indicate compatibility with all hardware generations, nor can it replace performance testing.

Tutorial for the Modelers Magic Music Community

Complete a basic task.

1. Register in the Modelers Magic Music community and create an API Key intended solely for use in a testing environment;
2. Select a model based on input type, context, quality, speed, and price;
3. First, call the model library to submit the minimal request and check the returned structure;
4. Test the stream output, parameters, and abnormal response using the model download method;
5. Record Tokens, number of calls, latency, error rate, and cost per call;
6. Move the key to the server-side key manager before integrating it into the actual application;

Create reusable professional workflows

1. Different keys and quotas are used for development, testing, and production environments;
2. Representative evaluation sets are established based on the model repository, the methods for downloading models, and the processes for uploading models and making contributions.
3. Set timeout, concurrency, retry, throttling, and budget limits;
4. Perform checks on the output regarding facts, security, format, and sensitive information;
5. Monitor changes in model version, price, latency, and failure rate;
6. Prepare plans for downgrading the model, implementing circuit breaking, and taking manual control;

Which users is it suitable for?

• Developers seeking domestic open-source models, datasets, and Ascend-compatible resources;
• Students who learn the processes of model loading, fine-tuning, inference, and deployment;
• Teams that need to collaborate using an organizational repository to maintain models and data;
• Researchers who wish to publish online demos, Notebooks, and the results of their models;
• Engineers who use PyTorch, MindSpore, or Ascend NPU.

Product advantages

• Models, datasets, experience spaces, and organizational collaboration are gathered in one community;
• It supports various download methods such as web pages, Git, Git LFS, and Hub Client;
• The openMind Library covers pre-training, fine-tuning, inference, and uploading;
• Compatible with PyTorch and MindSpore, with priority support for Ascend NPU;
• It is possible to conduct NPU availability tests and save logs as well as images;
• It has a community-oriented approach, and the barriers to accessing its resources are low.

Restrictions and Precautions

• Community warehouses are maintained by different contributors, and the model descriptions, quality, licenses, and security features vary from one another.
• The experience space may be in a dormant state or offline, and there is no unified publicly available price for online resource quotas; therefore, the demonstration environment cannot be regarded as a production SLA.
• Large files require GitLFS and sufficient disk space;
• Custom code poses supply chain risks;
• The NPU environment is sensitive to version combinations;
• Before downloading, training, and deploying, it is necessary to review the license, lock the version, scan the code, and create checksums and backups for the weights and data;

Frequently Asked Questions

Is the MoLe community free?

Models, datasets, documents, and access points for experiencing these services can be viewed for free; there is currently no unified membership fee. Online computing power, quotas for participating in activities, and enterprise services are subject to the specific rules outlined on relevant pages.

How to download Modelers models?

A single file can be downloaded from the website; it can also be cloned using Git and Git LFS, or the single file along with the complete snapshot can be obtained via the openMind Hub Client.

Which frameworks does the MoLe community support?

The openMind Library is compatible with PyTorch and MindSpore, and it is optimized particularly for the Ascend NPU; it can be used in conjunction with tools such as PEFT, DeepSpeed, Accelerate, and LLaMA Factory.

Can all community models be used for commercial purposes?

It cannot be generalized. Each model and dataset has different licensing terms, and it is necessary to check the restrictions regarding commercial use, attribution, licensing requests, and applicable scenarios in the specific repository.

Is Modelers open source?

Tools such as openMind Library and Hub Client come with open-source code, but the complete website and hosting services provided by the MoLe community cannot be considered fully open source.