无需搭建环境，零门槛带你体验Open-Sora文生视频应用
本文分享自华为云社区《Open-Sora 文生视频原来在AI Gallery上也能体验了》，作者：码上开花_Lancer。
体验链接：Open-Sora 文生视频案例体验
不久前，OpenAI Sora 凭借其惊人的视频生成效果迅速走红，在一堆文本转视频模型中脱颖而出，成为全球关注的焦点。之后，Colossal-AI团队又推出了新的开源解决方案“Open-Sora 1.0”，涵盖了整个训练过程，包括数据处理、所有训练细节和模型检查点，与世界各地的AI爱好者携手推进视频创作的新时代。
详细内容请参考：https://hpc-ai.com/blog/open-sora-v1.0
2024年4月份又更新了Open-Sora 1.1，它可以生成2s~15s，144p到720p分辨率的视频，支持文本到图像、文本到视频以及图像到视频的生成，让我们来看看Open-Sora 1.1的实际视频生成效果：
案例体验
🔹 本案例需使用 Pytorch-2.0.1 GPU-V100 及以上规格运行
🔹 点击Run in ModelArts，将会进入到ModelArts CodeLab中，这时需要你登录华为云账号，如果没有账号，则需要注册一个，且要进行实名认证，参考《如何创建华为云账号并且实名认证》 即可完成账号注册和实名认证。 登录之后，等待片刻，即可进入到CodeLab的运行环境
🔹 出现 Out Of Memory ，请检查是否为您的参数配置过高导致，修改参数配置，重启kernel或更换更高规格资源进行规避❗❗❗
1. 下载代码和模型
此处运行大约需要1分钟，请耐心等待！
import os
import moxing as mox
if not os.path.exists('Open-Sora'):
mox.file.copy_parallel('obs://modelbox-course/open-sora_1.1/Open-Sora', 'Open-Sora')
if not os.path.exists('/home/ma-user/.cache/huggingface'):
mox.file.copy_parallel('obs://modelbox-course/huggingface', '/home/ma-user/.cache/huggingface')
if not os.path.exists('Open-Sora/opensora/models/sd-vae-ft-ema'):
mox.file.copy_parallel('obs://modelbox-course/sd-vae-ft-ema', 'Open-Sora/opensora/models/sd-vae-ft-ema')
if not os.path.exists('Open-Sora/opensora/models/text_encoder/t5-v1_1-xxl'):
mox.file.copy_parallel('obs://modelbox-course/t5-v1_1-xxl', 'Open-Sora/opensora/models/text_encoder/t5-v1_1-xxl')
if not os.path.exists('/home/ma-user/work/t5.py'):
mox.file.copy_parallel('obs://modelbox-course/open-sora_1.1/t5.py', '/home/ma-user/work/t5.py')
if not os.path.exists('Open-Sora/opus-mt-zh-en'):
mox.file.copy_parallel('obs://modelarts-labs-bj4-v2/course/ModelBox/opus-mt-zh-en', 'Open-Sora/opus-mt-zh-en')
if not os.path.exists('/home/ma-user/work/frpc_linux_amd64'):
mox.file.copy_parallel('obs://modelarts-labs-bj4-v2/course/ModelBox/frpc_linux_amd64', '/home/ma-user/work/frpc_linux_amd64')INFO:root:Using MoXing-v2.1.6.879ab2f4-879ab2f4
INFO:root:List OBS time cost: 0.02 seconds.
INFO:root:Copy parallel total time cost: 41.71 seconds.
INFO:root:List OBS time cost: 0.14 seconds.
INFO:root:Copy parallel total time cost: 2.91 seconds.2. 配置运行环境
本案例依赖Python3.10.10及以上环境，因此我们首先创建虚拟环境：
!/home/ma-user/anaconda3/bin/conda clean -i
!/home/ma-user/anaconda3/bin/conda create -n python-3.10.10 python=3.10.10 -y --override-channels --channel https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
!/home/ma-user/anaconda3/envs/python-3.10.10/bin/pip install ipykernel/home/ma-user/anaconda3/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
/home/ma-user/anaconda3/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /home/ma-user/anaconda3/envs/python-3.10.10
added / updated specs:
- python=3.10.10The following packages will be downloaded:
package | build
---------------------------|-----------------
_libgcc_mutex-0.1 | main 3 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
_openmp_mutex-5.1 | 1_gnu 21 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
bzip2-1.0.8 | h5eee18b_6 262 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ca-certificates-2024.3.11 | h06a4308_0 127 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ld_impl_linux-64-2.38 | h1181459_1 654 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libffi-3.4.4 | h6a678d5_1 141 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgcc-ng-11.2.0 | h1234567_1 5.3 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libgomp-11.2.0 | h1234567_1 474 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libstdcxx-ng-11.2.0 | h1234567_1 4.7 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
libuuid-1.41.5 | h5eee18b_0 27 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
ncurses-6.4 | h6a678d5_0 914 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
openssl-1.1.1w | h7f8727e_0 3.7 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
pip-24.0 | py310h06a4308_0 2.7 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
python-3.10.10 | h7a1cb2a_2 26.9 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
readline-8.2 | h5eee18b_0 357 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
setuptools-69.5.1 | py310h06a4308_0 1012 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
sqlite-3.45.3 | h5eee18b_0 1.2 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tk-8.6.14 | h39e8969_0 3.4 MB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
tzdata-2024a | h04d1e81_0 116 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
wheel-0.43.0 | py310h06a4308_0 110 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
xz-5.4.6 | h5eee18b_1 643 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
zlib-1.2.13 | h5eee18b_1 111 KB https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main
------------------------------------------------------------
Total: 52.8 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex anaconda/pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex anaconda/pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
bzip2 anaconda/pkgs/main/linux-64::bzip2-1.0.8-h5eee18b_6
ca-certificates anaconda/pkgs/main/linux-64::ca-certificates-2024.3.11-h06a4308_0
ld_impl_linux-64 anaconda/pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi anaconda/pkgs/main/linux-64::libffi-3.4.4-h6a678d5_1
libgcc-ng anaconda/pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
libgomp anaconda/pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
libstdcxx-ng anaconda/pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
libuuid anaconda/pkgs/main/linux-64::libuuid-1.41.5-h5eee18b_0
ncurses anaconda/pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
openssl anaconda/pkgs/main/linux-64::openssl-1.1.1w-h7f8727e_0
pip anaconda/pkgs/main/linux-64::pip-24.0-py310h06a4308_0
python anaconda/pkgs/main/linux-64::python-3.10.10-h7a1cb2a_2
readline anaconda/pkgs/main/linux-64::readline-8.2-h5eee18b_0
setuptools anaconda/pkgs/main/linux-64::setuptools-69.5.1-py310h06a4308_0
sqlite anaconda/pkgs/main/linux-64::sqlite-3.45.3-h5eee18b_0
tk anaconda/pkgs/main/linux-64::tk-8.6.14-h39e8969_0
tzdata anaconda/pkgs/main/noarch::tzdata-2024a-h04d1e81_0
wheel anaconda/pkgs/main/linux-64::wheel-0.43.0-py310h06a4308_0
xz anaconda/pkgs/main/linux-64::xz-5.4.6-h5eee18b_1
zlib anaconda/pkgs/main/linux-64::zlib-1.2.13-h5eee18b_1Downloading and Extracting Packages
libffi-3.4.4 | 141 KB | ##################################### | 100%
_openmp_mutex-5.1 | 21 KB | ##################################### | 100%
xz-5.4.6 | 643 KB | ##################################### | 100%
tzdata-2024a | 116 KB | ##################################### | 100%
_libgcc_mutex-0.1 | 3 KB | ##################################### | 100%
zlib-1.2.13 | 111 KB | ##################################### | 100%
bzip2-1.0.8 | 262 KB | ##################################### | 100%
libuuid-1.41.5 | 27 KB | ##################################### | 100%
ca-certificates-2024 | 127 KB | ##################################### | 100%
libstdcxx-ng-11.2.0 | 4.7 MB | ##################################### | 100%
ncurses-6.4 | 914 KB | ##################################### | 100%
openssl-1.1.1w | 3.7 MB | ##################################### | 100%
wheel-0.43.0 | 110 KB | ##################################### | 100%
python-3.10.10 | 26.9 MB | ##################################### | 100%
pip-24.0 | 2.7 MB | ##################################### | 100%
readline-8.2 | 357 KB | ##################################### | 100%
tk-8.6.14 | 3.4 MB | ##################################### | 100%
setuptools-69.5.1 | 1012 KB | ##################################### | 100%
libgcc-ng-11.2.0 | 5.3 MB | ##################################### | 100%
ld_impl_linux-64-2.3 | 654 KB | ##################################### | 100%
libgomp-11.2.0 | 474 KB | ##################################### | 100%
sqlite-3.45.3 | 1.2 MB | ##################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
#
# To activate this environment, use
#
# $ conda activate python-3.10.10
#
# To deactivate an active environment, use
#
# $ conda deactivate
Looking in indexes: http://repo.myhuaweicloud.com/repository/pypi/simple
Collecting ipykernel
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/53/9d/40d5207db523363d9b5698f33778c18b0d591e3fdb6e0116b894b2a2491c/ipykernel-6.29.4-py3-none-any.whl (117 kB)
•[2K •[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━•[0m •[32m117.1/117.1 kB•[0m •[31m10.6 MB/s•[0m eta •[36m0:00:00•[0m
......
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/80/03/6ea8b1b2a5ab40a7a60dc464d3daa7aa546e0a74d74a9f8ff551ea7905db/executing-2.0.1-py2.py3-none-any.whl (24 kB)
Collecting asttokens>=2.1.0 (from stack-data->ipython>=7.23.1->ipykernel)
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/45/86/4736ac618d82a20d87d2f92ae19441ebc7ac9e7a581d7e58bbe79233b24a/asttokens-2.4.1-py2.py3-none-any.whl (27 kB)
Collecting pure-eval (from stack-data->ipython>=7.23.1->ipykernel)
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/2b/27/77f9d5684e6bce929f5cfe18d6cfbe5133013c06cb2fbf5933670e60761d/pure_eval-0.2.2-py3-none-any.whl (11 kB)
Installing collected packages: wcwidth, pure-eval, ptyprocess, typing-extensions, traitlets, tornado, six, pyzmq, pygments, psutil, prompt-toolkit, platformdirs, pexpect, parso, packaging, nest-asyncio, executing, exceptiongroup, decorator, debugpy, python-dateutil, matplotlib-inline, jupyter-core, jedi, comm, asttokens, stack-data, jupyter-client, ipython, ipykernel
Successfully installed asttokens-2.4.1 comm-0.2.2 debugpy-1.8.1 decorator-5.1.1 exceptiongroup-1.2.1 executing-2.0.1 ipykernel-6.29.4 ipython-8.25.0 jedi-0.19.1 jupyter-client-8.6.2 jupyter-core-5.7.2 matplotlib-inline-0.1.7 nest-asyncio-1.6.0 packaging-24.0 parso-0.8.4 pexpect-4.9.0 platformdirs-4.2.2 prompt-toolkit-3.0.46 psutil-5.9.8 ptyprocess-0.7.0 pure-eval-0.2.2 pygments-2.18.0 python-dateutil-2.9.0.post0 pyzmq-26.0.3 six-1.16.0 stack-data-0.6.3 tornado-6.4 traitlets-5.14.3 typing-extensions-4.12.1 wcwidth-0.2.13import json
import os
data = {
"display_name": "python-3.10.10",
"env": {
"PATH": "/home/ma-user/anaconda3/envs/python-3.10.10/bin:/home/ma-user/anaconda3/envs/python-3.7.10/bin:/modelarts/authoring/notebook-conda/bin:/opt/conda/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/ma-user/modelarts/ma-cli/bin:/home/ma-user/modelarts/ma-cli/bin:/home/ma-user/anaconda3/envs/PyTorch-1.8/bin"
},
"language": "python",
"argv": [
"/home/ma-user/anaconda3/envs/python-3.10.10/bin/python",
"-m",
"ipykernel",
"-f",
"{connection_file}"
]
}
if not os.path.exists("/home/ma-user/anaconda3/share/jupyter/kernels/python-3.10.10/"):
os.mkdir("/home/ma-user/anaconda3/share/jupyter/kernels/python-3.10.10/")
with open('/home/ma-user/anaconda3/share/jupyter/kernels/python-3.10.10/kernel.json', 'w') as f:
json.dump(data, f, indent=4)conda env list/home/ma-user/anaconda3/lib/python3.7/site-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.15) or chardet (3.0.4) doesn't match a supported version!
RequestsDependencyWarning)
# conda environments:
#
base * /home/ma-user/anaconda3
python-3.10.10 /home/ma-user/anaconda3/envs/python-3.10.10
python-3.7.10 /home/ma-user/anaconda3/envs/python-3.7.10Note: you may need to restart the kernel to use updated packages.创建完成后，稍等片刻，或刷新页面，点击右上角kernel选择python-3.10.10
查看Python版本
!python -VPython 3.10.10检查可用GPU，至少需要32GB显存
!nvidia-smiWed Jun 5 16:22:37 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.57.02 Driver Version: 470.57.02 CUDA Version: 11.4 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla V100-PCIE... On | 00000000:00:0D.0 Off | 0 |
| N/A 28C P0 25W / 250W | 0MiB / 32510MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+安装依赖包
!pip install --upgrade pip
!pip install torch==2.0.1 torchvision==0.15.2 torchaudio==2.0.2 xformers==0.0.22Looking in indexes: http://repo.myhuaweicloud.com/repository/pypi/simple
Requirement already satisfied: pip in /home/ma-user/anaconda3/envs/python-3.10.10/lib/python3.10/site-packages (24.0)
Looking in indexes: http://repo.myhuaweicloud.com/repository/pypi/simple
Collecting torch==2.0.1
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/8c/4d/17e07377c9c3d1a0c4eb3fde1c7c16b5a0ce6133ddbabc08ceef6b7f2645/torch-2.0.1-cp310-cp310-manylinux1_x86_64.whl (619.9 MB)
•[2K •[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━•[0m •[32m619.9/619.9 MB•[0m •[31m8.2 MB/s•[0m eta •[36m0:00:00•[0m00:01•[0m00:01•[0m
•[?25hCollecting torchvision==0.15.2
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/87/0f/88f023bf6176d9af0f85feedf4be129f9cf2748801c4d9c690739a10c100/torchvision-0.15.2-cp310-cp310-manylinux1_x86_64.whl (6.0 MB)
•[2K •[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━•[0m •[32m6.0/6.0 MB•[0m •[31m109.5 MB/s•[0m eta •[36m0:00:00•[0ma •[36m0:00:01•[0m
•[?25hCollecting torchaudio==2.0.2
Downloading
•[?25hCollecting certifi>=2017.4.17 (from requests->torchvision==0.15.2)
Downloading http://repo.myhuaweicloud.com/repository/pypi/packages/5b/11/1e78951465b4a225519b8c3ad29769c49e0d8d157a070f681d5b6d64737f/certifi-2024.6.2-py3-none-any.whl (164 kB)
•[2K •[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━