javascript - How to add Speech-to-Text into a chat application that is written on React Native

Question

My friends and I are working on a project that is a chat application that will implement Speech-To-Text, Text-to-Speech, and Translation APIs of Google. Gifted Chat and Firebase on the chat application. Chat App is working well with Firebase. We added TTS on it and it is also working well but we can't add the STT. We aim that users can use a microphone and the app can convert that speech into text. This text will automatically appear on the text box of the user. We believe that we must manually add STT to Gifted Chats modules but we don't know how to do it. There is also no source on the Internet about that. We will be so happy if anyone can help us. Thank you!

Answer 1

使用 react-native-voice 库进行语音识别

Gifted Chat 中有一个 text 和 onInputTextChanged 属性，可以帮助你处理文本框中出现的语音结果。

import Voice from '@react-native-community/voice';
import React, { useState, useEffect } from 'react';
import { GiftedChat, Composer } from 'react-native-gifted-chat';

const VoiceTest = () => {
  const [speech, setSpeech] = useState('');

  useEffect(() => {
    Voice.onSpeechStart = onSpeechStart;
    Voice.onSpeechEnd = onSpeechEnd;
    Voice.onSpeechError = onSpeechError;
    Voice.onSpeechResults = onSpeechResults;
    Voice.onSpeechPartialResults = onSpeechPartialResults;
  
  return () => Voice.destroy().then(Voice.removeAllListeners)
}, [])

  const onSpeechStart = () => {
  ...
  }

  const onSpeechEnd = () => {
  ...
  }

  const onSpeechError = () => {
  ...
  }

  const onSpeechResults = (e) => {
  ...
  setSpeech(e.value[0])
  }

  const onSpeechPartialResults = (e) => {
  ...
  setSpeech(e.value[0])
  }

  const startListening = () => {
  ...
  // You can set the locale to any language you want it to recognize, I am using Nigerian English.
  Voice.start('en-NG')
  }

  const renderComposer = (props) => {
  // Adds a Mic Button in the text box, you can style it as you want
  return (
    <View style={{ flexDirection: 'row' }}>
     <Composer {...props} />
     <MicrophoneButton onPress={startListening} />
    </View>
   )
  }

  return (
    ...
    <GiftedChat 
      renderComposer={renderComposer}
      text={speech}
      onInputTextChanged={setSpeech} 
     />
    ...
  )
 ...

Answer 2

您可以使用react-native-voice库

这是示例用法：

import Voice from '@react-native-community/voice';
import React, {Component} from 'react';

class VoiceTest extends Component {
  constructor(props) {
    Voice.onSpeechStart = this.onSpeechStartHandler.bind(this);
    Voice.onSpeechEnd = this.onSpeechEndHandler.bind(this);
    Voice.onSpeechResults = this.onSpeechResultsHandler.bind(this);
  }
  onStartButtonPress(e){
    Voice.start('en-US');
  }
  ...
}