摘要翻译：
音色空间在音乐知觉中被用来研究基于不同等级的乐器之间的知觉关系。然而，这些空间不能推广到新的例子，也不能提供可逆映射，从而阻止音频合成。同时，生成模型的目标是提供合成新音色的方法。然而，这些系统不提供对其内部工作的理解，通常与任何知觉相关的信息无关。在这里，我们表明变分自动编码器(VAE)可以通过构造生成音色空间来缓解所有这些限制。为此，我们使用VAEs来学习一个音频潜在空间，同时使用音色研究中的感知评级来规则化这个空间的组织。由此产生的空间允许我们分析新的乐器，同时能够从这个空间的任何一点合成音频。我们引入了一个特定的正则化，允许在这些空间上执行任何给定的相似距离。我们表明，所得到的空间提供了与音色空间几乎相似的距离关系。我们评估了几种谱变换，并表明非平稳Gabor变换(NSGT)提供了最高的音色空间相关性和最佳的合成质量。此外，我们还表明这些空间可以推广到新的乐器，并可以生成乐器之间的任何路径来理解它们的音色关系。由于这些空间是连续的，我们研究了音频描述符如何沿着潜在维度的行为。我们证明，即使描述子具有整体非线性拓扑结构，它们也遵循局部光滑的演化。在此基础上，我们提出了一种基于描述符的合成方法，并说明了在保持乐器音色结构的前提下，可以控制乐器的描述符。
---
英文标题：
《Generative timbre spaces: regularizing variational auto-encoders with
  perceptual metrics》
---
作者：
Philippe Esling, Axel Chemla--Romeu-Santos, Adrien Bitton
---
最新提交年份：
2018
---
分类信息：

一级分类：Computer Science        计算机科学
二级分类：Sound        声音
分类描述：Covers all aspects of computing with sound, and sound as an information channel. Includes models of sound, analysis and synthesis, audio user interfaces, sonification of data, computer music, and sound signal processing. Includes ACM Subject Class H.5.5, and intersects with H.1.2, H.5.1, H.5.2, I.2.7, I.5.4, I.6.3, J.5, K.4.2.
涵盖了声音计算的各个方面，以及声音作为一种信息通道。包括声音模型、分析和合成、音频用户界面、数据的可听化、计算机音乐和声音信号处理。包括ACM学科类H.5.5，并与H.1.2、H.5.1、H.5.2、I.2.7、I.5.4、I.6.3、J.5、K.4.2交叉。
--
一级分类：Electrical Engineering and Systems Science        电气工程与系统科学
二级分类：Audio and Speech Processing        音频和语音处理
分类描述：Theory and methods for processing signals representing audio, speech, and language, and their applications. This includes analysis, synthesis, enhancement, transformation, classification and interpretation of such signals as well as the design, development, and evaluation of associated signal processing systems. Machine learning and pattern analysis applied to any of the above areas is also welcome.  Specific topics of interest include: auditory modeling and hearing aids; acoustic beamforming and source localization; classification of acoustic scenes; speaker separation; active noise control and echo cancellation; enhancement; de-reverberation; bioacoustics; music signals analysis, synthesis and modification; music information retrieval;  audio for multimedia and joint audio-video processing; spoken and written language modeling, segmentation, tagging, parsing, understanding, and translation; text mining; speech production, perception, and psychoacoustics; speech analysis, synthesis, and perceptual modeling and coding; robust speech recognition; speaker recognition and characterization; deep learning, online learning, and graphical models applied to speech, audio, and language signals; and implementation aspects ranging from system architecture to fast algorithms.
处理代表音频、语音和语言的信号的理论和方法及其应用。这包括分析、合成、增强、转换、分类和解释这些信号，以及相关信号处理系统的设计、开发和评估。机器学习和模式分析应用于上述任何领域也是受欢迎的。感兴趣的具体主题包括：听觉建模和助听器；声波束形成与声源定位；声场景分类；说话人分离；有源噪声控制和回声消除；增强；去混响；生物声学；音乐信号的分析、合成与修饰；音乐信息检索；多媒体音频和联合音视频处理；口语和书面语建模、切分、标注、句法分析、理解和翻译；文本挖掘；言语产生、感知和心理声学；语音分析、合成、感知建模和编码；鲁棒语音识别；说话人识别与特征描述；应用于语音、音频和语言信号的深度学习、在线学习和图形模型；以及从系统架构到快速算法的实现方面。
--

---
英文摘要：
  Timbre spaces have been used in music perception to study the perceptual relationships between instruments based on dissimilarity ratings. However, these spaces do not generalize to novel examples and do not provide an invertible mapping, preventing audio synthesis. In parallel, generative models have aimed to provide methods for synthesizing novel timbres. However, these systems do not provide an understanding of their inner workings and are usually not related to any perceptually relevant information. Here, we show that Variational Auto-Encoders (VAE) can alleviate all of these limitations by constructing generative timbre spaces. To do so, we adapt VAEs to learn an audio latent space, while using perceptual ratings from timbre studies to regularize the organization of this space. The resulting space allows us to analyze novel instruments, while being able to synthesize audio from any point of this space. We introduce a specific regularization allowing to enforce any given similarity distances onto these spaces. We show that the resulting space provide almost similar distance relationships as timbre spaces. We evaluate several spectral transforms and show that the Non-Stationary Gabor Transform (NSGT) provides the highest correlation to timbre spaces and the best quality of synthesis. Furthermore, we show that these spaces can generalize to novel instruments and can generate any path between instruments to understand their timbre relationships. As these spaces are continuous, we study how audio descriptors behave along the latent dimensions. We show that even though descriptors have an overall non-linear topology, they follow a locally smooth evolution. Based on this, we introduce a method for descriptor-based synthesis and show that we can control the descriptors of an instrument while keeping its timbre structure.
---
PDF链接：
https://arxiv.org/pdf/1805.08501