使用 WebRTC、React 和 NestJS 构建视频聊天应用程序

在当今万物互联的世界里,实时通信应用已成为个人和专业应用的必需品。尤其是视频聊天应用程序,其受欢迎程度呈指数级增长,而全球性事件导致远程互动的需求也加速了这一趋势。

在本综合指南中,我们将介绍如何使用下面相关技术构建一个全功能的视频聊天应用程序:

  • WebRTC(Web Real-Time Communication)用于点对点音频/视频流
  • React 打造响应式交互前端的
  • NestJS 用于稳健、可扩展的后端
  • Socket.IO 用于对等方(peers)之间的信号传输
使用 WebRTC、React 和 NestJS 构建视频聊天应用程序

项目架构

该视频聊天应用程序将采用以下架构:

  • 捕获媒体流并显示视频的 React 前端
  • 通过 Socket.IO 处理信令的 NestJS 后端
  • WebRTC 用于在信令后建立直接对等连接

信令流程

  • 用户 A 加入一个房间
  • 用户 B 加入同一房间
  • 服务器通知双方用户
  • 用户 A 创建 offer(SDP)并通过服务器发送给用户 B
  • 用户 B 接收 offer ,创建 answer 并将其发送回用户 A
  • 两个用户交换 ICE 候选以建立最佳连接路径
  • 建立点对点直接连接

前提条件

在开始之前,请确保您已安装:

  • Node.js(v14 或更高版本)
  • npm 或 yarn
  • 支持 WebRTC 的现代 Web 浏览器(Chrome、Firefox、Safari、Edge)

设置后端(NestJS)

首先从创建 NestJS 后端开始,它将处理用户身份验证和信令。

1. 安装 NestJS CLI 并创建项目:

npm i -g @nestjs/cli
nest new video-chat-backend
cd video-chat-backend

2. 安装所需的依赖项:

npm install @nestjs/websockets @nestjs/platform-socket.io socket.io uuid

3. 创建 WebSocket 网关

创建新文件 src/signaling/signaling.gateway.ts:typescript

import {
  WebSocketGateway,
  SubscribeMessage,
  MessageBody,
  WebSocketServer,
  ConnectedSocket,
} from '@nestjs/websockets';
import { Server, Socket } from 'socket.io';
import { v4 as uuidv4 } from 'uuid';

interface Room {
  id: string;
  name: string;
  users: string[];
}

@WebSocketGateway({
  cors: {
    origin: '*',
  },
})
export class SignalingGateway {
  @WebSocketServer()
  server: Server;

  private rooms: Room[] = [];
  private socketToRoom = new Map<string, string>();

  @SubscribeMessage('join-room')
  handleJoinRoom(
    @ConnectedSocket() client: Socket,
    @MessageBody() data: { roomName: string; userName: string },
  ) {
    const { roomName, userName } = data;
    
    // 查找房间,如果不存在则创建
    let room = this.rooms.find((r) => r.name === roomName);
    if (!room) {
      room = { id: uuidv4(), name: roomName, users: [] };
      this.rooms.push(room);
    }
    
    // 将用户添加到房间
    room.users.push(client.id);
    this.socketToRoom.set(client.id, room.id);
    
    // 加入socket.io房间
    client.join(room.id);
    
    // 通知房间中的其他用户
    client.to(room.id).emit('user-joined', {
      userId: client.id,
      userName,
    });
    
    // 将现有用户列表发送给新用户
    const usersInRoom = room.users.filter((id) => id !== client.id);
    client.emit('room-users', usersInRoom);
    
    console.log(`User ${userName} (${client.id}) joined room ${roomName}`);
  }

  @SubscribeMessage('offer')
  handleOffer(
    @ConnectedSocket() client: Socket,
    @MessageBody() data: { target: string; offer: any; caller: string },
  ) {
    const { target, offer, caller } = data;
    this.server.to(target).emit('offer', { offer, caller });
  }

  @SubscribeMessage('answer')
  handleAnswer(
    @ConnectedSocket() client: Socket,
    @MessageBody() data: { target: string; answer: any },
  ) {
    const { target, answer } = data;
    this.server.to(target).emit('answer', { answer, answerer: client.id });
  }

  @SubscribeMessage('ice-candidate')
  handleIceCandidate(
    @ConnectedSocket() client: Socket,
    @MessageBody() data: { target: string; candidate: any },
  ) {
    const { target, candidate } = data;
    this.server.to(target).emit('ice-candidate', {
      candidate,
      sender: client.id,
    });
  }

  handleDisconnect(client: Socket) {
    // 查找用户所在的房间
    const roomId = this.socketToRoom.get(client.id);
    if (roomId) {
      // 查找房间
      const roomIndex = this.rooms.findIndex((r) => r.id === roomId);
      if (roomIndex >= 0) {
        // 从房间中移除用户
        const room = this.rooms[roomIndex];
        room.users = room.users.filter((id) => id !== client.id);
        
        // 如果房间为空则移除
        if (room.users.length === 0) {
          this.rooms.splice(roomIndex, 1);
        } else {
          // 通知其他用户连接断开
          client.to(roomId).emit('user-disconnected', client.id);
        }
      }
      
      // 从 mapping 中移除
      this.socketToRoom.delete(client.id);
    }
    
    console.log(`User ${client.id} disconnected`);
  }
}

4. 为 WebSocket 网关创建模块

创建一个新文件src/signaling/signaling.module.ts

import { Module } from '@nestjs/common';
import { SignalingGateway } from './signaling.gateway';

@Module({
  providers: [SignalingGateway],
})
export class SignalingModule {}

5. 更新应用程序模块

更新 src/app.module.ts,加入 SignalingModule:

import { Module } from '@nestjs/common';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { SignalingModule } from './signaling/signaling.module';

@Module({
  imports: [SignalingModule],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

设置前端(React)

现在,开始创建 React 前端应用程序。

1. 创建新的 React 项目

npx create-react-app video-chat-frontend --template typescript
cd video-chat-frontend

2. 安装所需的依赖项

npm install socket.io-client styled-components @types/styled-components

3. 设置 Socket.IO 连接

创建一个新文件src/services/socket.ts

import { io } from 'socket.io-client';

const SOCKET_SERVER_URL = 'http://localhost:3001';

export const socket = io(SOCKET_SERVER_URL);

export const joinRoom = (roomName: string, userName: string) => {
  socket.emit('join-room', { roomName, userName });
};

export const sendOffer = (target: string, offer: any, caller: string) => {
  socket.emit('offer', { target, offer, caller });
};

export const sendAnswer = (target: string, answer: any) => {
  socket.emit('answer', { target, answer });
};

export const sendIceCandidate = (target: string, candidate: any) => {
  socket.emit('ice-candidate', { target, candidate });
};

4. 创建 WebRTC 服务

创建一个新文件src/services/webrtc.ts

const ICE_SERVERS = {
  iceServers: [
    { urls: 'stun:stun.l.google.com:19302' },
    { urls: 'stun:stun1.l.google.com:19302' },
  ],
};

export class PeerConnection {
  private peerConnection: RTCPeerConnection;
  private localStream: MediaStream | null = null;
  private remoteStream: MediaStream;
  private onIceCandidateCallback: (candidate: RTCIceCandidate) => void;
  private onTrackCallback: (stream: MediaStream) => void;

  constructor(
    onIceCandidate: (candidate: RTCIceCandidate) => void,
    onTrack: (stream: MediaStream) => void
  ) {
    this.peerConnection = new RTCPeerConnection(ICE_SERVERS);
    this.remoteStream = new MediaStream();
    this.onIceCandidateCallback = onIceCandidate;
    this.onTrackCallback = onTrack;

    // 设置事件处理程序
    this.peerConnection.onicecandidate = (event) => {
      if (event.candidate) {
        this.onIceCandidateCallback(event.candidate);
      }
    };

    this.peerConnection.ontrack = (event) => {
      event.streams[0].getTracks().forEach((track) => {
        this.remoteStream.addTrack(track);
      });
      this.onTrackCallback(this.remoteStream);
    };
  }

  async setLocalStream(stream: MediaStream) {
    this.localStream = stream;
    stream.getTracks().forEach((track) => {
      if (this.localStream) {
        this.peerConnection.addTrack(track, this.localStream);
      }
    });
  }

  async createOffer() {
    try {
      const offer = await this.peerConnection.createOffer();
      await this.peerConnection.setLocalDescription(offer);
      return offer;
    } catch (error) {
      console.error('Error creating offer', error);
      throw error;
    }
  }

  async createAnswer(offer: RTCSessionDescriptionInit) {
    try {
      await this.peerConnection.setRemoteDescription(new RTCSessionDescription(offer));
      const answer = await this.peerConnection.createAnswer();
      await this.peerConnection.setLocalDescription(answer);
      return answer;
    } catch (error) {
      console.error('Error creating answer', error);
      throw error;
    }
  }

  async setRemoteAnswer(answer: RTCSessionDescriptionInit) {
    try {
      await this.peerConnection.setRemoteDescription(new RTCSessionDescription(answer));
    } catch (error) {
      console.error('Error setting remote description', error);
      throw error;
    }
  }

  addIceCandidate(candidate: RTCIceCandidateInit) {
    try {
      this.peerConnection.addIceCandidate(new RTCIceCandidate(candidate));
    } catch (error) {
      console.error('Error adding ICE candidate', error);
      throw error;
    }
  }

  close() {
    this.peerConnection.close();
    if (this.localStream) {
      this.localStream.getTracks().forEach((track) => track.stop());
    }
  }
}

export const getUserMedia = async () => {
  try {
    const stream = await navigator.mediaDevices.getUserMedia({
      video: true,
      audio: true,
    });
    return stream;
  } catch (error) {
    console.error('Error accessing media devices', error);
    throw error;
  }
};

创建 React 组件

开始创建视频聊天应用程序的主要组件。

房间组件

创建 src/components/Room.tsx

import React, { useEffect, useRef, useState } from 'react';
import styled from 'styled-components';
import { socket, joinRoom, sendOffer, sendAnswer, sendIceCandidate } from '../services/socket';
import { PeerConnection, getUserMedia } from '../services/webrtc';

const Container = styled.div`
  display: flex;
  flex-direction: column;
  align-items: center;
  padding: 20px;
`;

const VideoContainer = styled.div`
  display: flex;
  flex-wrap: wrap;
  justify-content: center;
  gap: 20px;
  margin-top: 20px;
`;

const VideoWrapper = styled.div`
  width: 400px;
  position: relative;
  border-radius: 10px;
  overflow: hidden;
  box-shadow: 0 4px 8px rgba(0, 0, 0, 0.2);
`;

const Video = styled.video`
  width: 100%;
  height: 300px;
  object-fit: cover;
  background-color: #1a1a1a;
`;

const UserLabel = styled.div`
  position: absolute;
  bottom: 10px;
  left: 10px;
  background-color: rgba(0, 0, 0, 0.5);
  color: white;
  padding: 5px 10px;
  border-radius: 5px;
  font-size: 14px;
`;

const Controls = styled.div`
  display: flex;
  justify-content: center;
  margin-top: 20px;
  gap: 10px;
`;

const Button = styled.button`
  padding: 10px 15px;
  border-radius: 5px;
  border: none;
  background-color: #4a4a4a;
  color: white;
  cursor: pointer;
  transition: background-color 0.3s;

  &:hover {
    background-color: #666;
  }
  
  &.leave {
    background-color: #e74c3c;
  }
  
  &.leave:hover {
    background-color: #c0392b;
  }
`;

interface RoomProps {
  roomName: string;
  userName: string;
  onLeaveRoom: () => void;
}

interface Peer {
  id: string;
  name?: string;
  connection: PeerConnection;
  stream?: MediaStream;
}

const Room: React.FC<RoomProps> = ({ roomName, userName, onLeaveRoom }) => {
  const [peers, setPeers] = useState<Map<string, Peer>>(new Map());
  const [localStream, setLocalStream] = useState<MediaStream | null>(null);
  const localVideoRef = useRef<HTMLVideoElement>(null);
  
  //  初始化媒体并加入房间
  useEffect(() => {
    const initializeRoom = async () => {
      try {
        // 获取本地媒体
        const stream = await getUserMedia();
        setLocalStream(stream);
        
        if (localVideoRef.current) {
          localVideoRef.current.srcObject = stream;
        }
        
        // 加入房间
        joinRoom(roomName, userName);
      } catch (error) {
        console.error('Error initializing room:', error);
      }
    };
    
    initializeRoom();
    
    // 清理组件卸载
    return () => {
      if (localStream) {
        localStream.getTracks().forEach(track => track.stop());
      }
      
      // 关闭所有对等连接
      peers.forEach(peer => {
        peer.connection.close();
      });
      
      socket.off();
    };
  }, []);
  
  // Socket 事件监听器
  useEffect(() => {
    // 当新用户加入房间时
    socket.on('user-joined', async ({ userId, userName: peerName }) => {
      console.log(`User ${peerName} (${userId}) joined the room`);
      
      if (localStream) {
        // 创建新的 peer 连接
        const peerConnection = new PeerConnection(
          (candidate) => {
            sendIceCandidate(userId, candidate);
          },
          (stream) => {
            setPeers(prev => {
              const updated = new Map(prev);
              const peer = updated.get(userId);
              if (peer) {
                peer.stream = stream;
                updated.set(userId, peer);
              }
              return updated;
            });
          }
        );
        
        await peerConnection.setLocalStream(localStream);
        
        // 创建 offer
        const offer = await peerConnection.createOffer();
        sendOffer(userId, offer, socket.id);
        
        // 添加到 peers list 
        setPeers(prev => {
          const updated = new Map(prev);
          updated.set(userId, {
            id: userId,
            name: peerName,
            connection: peerConnection
          });
          return updated;
        });
      }
    });
    
    // 当接收到房间中已有用户列表时
    socket.on('room-users', (userIds: string[]) => {
      console.log('Users in room:', userIds);
    });
    
    // 当接收到来自另一个 peer 的 offer 时
    socket.on('offer', async ({ offer, caller }) => {
      console.log('Received offer from:', caller);
      
      if (localStream) {
        // 创建新的 peer 连接
        const peerConnection = new PeerConnection(
          (candidate) => {
            sendIceCandidate(caller, candidate);
          },
          (stream) => {
            setPeers(prev => {
              const updated = new Map(prev);
              const peer = updated.get(caller);
              if (peer) {
                peer.stream = stream;
                updated.set(caller, peer);
              }
              return updated;
            });
          }
        );
        
        await peerConnection.setLocalStream(localStream);
        
        // 创建 answer
        const answer = await peerConnection.createAnswer(offer);
        sendAnswer(caller, answer);
        
        // 添加到 peers 列表
        setPeers(prev => {
          const updated = new Map(prev);
          updated.set(caller, {
            id: caller,
            connection: peerConnection
          });
          return updated;
        });
      }
    });
    
    // 当收到 answer 时
    socket.on('answer', async ({ answer, answerer }) => {
      console.log('Received answer from:', answerer);
      
      const peer = peers.get(answerer);
      if (peer) {
        await peer.connection.setRemoteAnswer(answer);
      }
    });
    
    // 当收到 ICE candidate 时
    socket.on('ice-candidate', ({ candidate, sender }) => {
      console.log('Received ICE candidate from:', sender);
      
      const peer = peers.get(sender);
      if (peer) {
        peer.connection.addIceCandidate(candidate);
      }
    });
    
    // 当用户断开连接时
    socket.on('user-disconnected', (userId) => {
      console.log('User disconnected:', userId);
      
      const peer = peers.get(userId);
      if (peer) {
        peer.connection.close();
        setPeers(prev => {
          const updated = new Map(prev);
          updated.delete(userId);
          return updated;
        });
      }
    });
    
    return () => {
      socket.off('user-joined');
      socket.off('room-users');
      socket.off('offer');
      socket.off('answer');
      socket.off('ice-candidate');
      socket.off('user-disconnected');
    };
  }, [peers, localStream, roomName]);
  
  const handleLeaveRoom = () => {
    // 停止所有tracks
    if (localStream) {
      localStream.getTracks().forEach(track => track.stop());
    }
    
    // 关闭所有 peer 连接
    peers.forEach(peer => {
      peer.connection.close();
    });
    
    // 断开 socket
    socket.disconnect();
    
    // 调用父组件回调
    onLeaveRoom();
  };
  
  const toggleMute = () => {
    if (localStream) {
      const audioTracks = localStream.getAudioTracks();
      audioTracks.forEach(track => {
        track.enabled = !track.enabled;
      });
    }
  };
  
  const toggleVideo = () => {
    if (localStream) {
      const videoTracks = localStream.getVideoTracks();
      videoTracks.forEach(track => {
        track.enabled = !track.enabled;
      });
    }
  };
  
  return (
    <Container>
      <h2>Room: {roomName}</h2>
      
      <VideoContainer>
        {/* Local video */}
        <VideoWrapper>
          <Video ref={localVideoRef} autoPlay muted playsInline />
          <UserLabel>You ({userName})</UserLabel>
        </VideoWrapper>
        
        {/* Remote videos */}
        {Array.from(peers.values()).map(peer => (
          peer.stream && (
            <VideoWrapper key={peer.id}>
              <Video
                autoPlay
                playsInline
                ref={el => {
                  if (el && peer.stream) {
                    el.srcObject = peer.stream;
                  }
                }}
              />
              <UserLabel>{peer.name || 'Peer'}</UserLabel>
            </VideoWrapper>
          )
        ))}
      </VideoContainer>
      
      <Controls>
        <Button onClick={toggleMute}>Toggle Mute</Button>
        <Button onClick={toggleVideo}>Toggle Video</Button>
        <Button className="leave" onClick={handleLeaveRoom}>Leave Room</Button>
      </Controls>
    </Container>
  );
};

export default Room;

主页组件

创建 src/components/Home.tsx

import React, { useState } from 'react';
import styled from 'styled-components';

const Container = styled.div`
  display: flex;
  flex-direction: column;
  align-items: center;
  justify-content: center;
  min-height: 100vh;
  background-color: #f5f5f5;
  padding: 20px;
`;

const FormContainer = styled.div`
  background-color: white;
  border-radius: 10px;
  padding: 30px;
  box-shadow: 0 4px 8px rgba(0, 0, 0, 0.1);
  width: 100%;
  max-width: 400px;
`;

const Title = styled.h1`
  text-align: center;
  color: #333;
  margin-bottom: 30px;
`;

const Form = styled.form`
  display: flex;
  flex-direction: column;
  gap: 20px;
`;

const InputGroup = styled.div`
  display: flex;
  flex-direction: column;
  gap: 5px;
`;

const Label = styled.label`
  font-weight: bold;
  color: #555;
`;

const Input = styled.input`
  padding: 10px;
  border: 1px solid #ddd;
  border-radius: 5px;
  font-size: 16px;
`;

const Button = styled.button`
  padding: 12px;
  background-color: #4285f4;
  color: white;
  border: none;
  border-radius: 5px;
  font-size: 16px;
  cursor: pointer;
  transition: background-color 0.3s;

  &:hover {
    background-color: #3367d6;
  }
`;

interface HomeProps {
  onJoinRoom: (roomName: string, userName: string) => void;
}

const Home: React.FC<HomeProps> = ({ onJoinRoom }) => {
  const [roomName, setRoomName] = useState('');
  const [userName, setUserName] = useState('');

  const handleSubmit = (e: React.FormEvent) => {
    e.preventDefault();
    if (roomName && userName) {
      onJoinRoom(roomName, userName);
    }
  };

  return (
    <Container>
      <FormContainer>
        <Title>Join Video Chat</Title>
        <Form onSubmit={handleSubmit}>
          <InputGroup>
            <Label htmlFor="userName">Your Name</Label>
            <Input
              id="userName"
              type="text"
              value={userName}
              onChange={(e) => setUserName(e.target.value)}
              placeholder="Enter your name"
              required
            />
          </InputGroup>
          <InputGroup>
            <Label htmlFor="roomName">Room Name</Label>
            <Input
              id="roomName"
              type="text"
              value={roomName}
              onChange={(e) => setRoomName(e.target.value)}
              placeholder="Enter room name"
              required
            />
          </InputGroup>
          <Button type="submit">Join Room</Button>
        </Form>
      </FormContainer>
    </Container>
  );
};

export default Home;

更新应用程序组件

更新src/App.tsx

import React, { useState } from 'react';
import Home from './components/Home';
import Room from './components/Room';

const App: React.FC = () => {
  const [isInRoom, setIsInRoom] = useState(false);
  const [roomName, setRoomName] = useState('');
  const [userName, setUserName] = useState('');

  const handleJoinRoom = (room: string, user: string) => {
    setRoomName(room);
    setUserName(user);
    setIsInRoom(true);
  };

  const handleLeaveRoom = () => {
    setIsInRoom(false);
  };

  return (
    <div className="App">
      {isInRoom ? (
        <Room
          roomName={roomName}
          userName={userName}
          onLeaveRoom={handleLeaveRoom}
        />
      ) : (
        <Home onJoinRoom={handleJoinRoom} />
      )}
    </div>
  );
};

export default App;

更新 CSS

最后,添加一些全局样式src/index.css

* {
  box-sizing: border-box;
  margin: 0;
  padding: 0;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
    'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
    sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  background-color: #f0f2f5;
}

h1, h2, h3, h4, h5, h6 {
  margin-bottom: 0.5rem;
}

运行应用程序

现在可以运行后端和前端应用程序:

运行后端

cd video-chat-backend
npm run start:dev

运行前端

cd video-chat-frontend
npm start

在浏览器中访问 http://localhost:3000 访问应用程序。在另一个浏览器窗口或设备上打开网站,测试视频聊天功能。

常见问题和解决方案

1. ICE 连接失败

如果您的对等设备无法建立连接,可能是由于 NAT 穿越问题。考虑在配置中添加更多 STUN/TURN 服务器:

const  ICE_SERVERS = { 
  iceServers : [ 
    { urls : 'stun:stun.l.google.com:19302' }, 
    { urls : 'stun:stun1.l.google.com:19302' }, 
    // 添加更多 STUN 服务器
    // 添加 TURN 服务器(用于后备)
     { 
      urls : 'turn:your-turn-server.com:3478' , 
      username : 'username' , 
      credential : 'password'
     } 
  ], 
};

2. 设备权限问题

请求媒体设备时始终处理错误:

try {
  const stream = await navigator.mediaDevices.getUserMedia({
    video: true,
    audio: true,
  });
  // 使用流
} catch (error) {
  if (error.name === 'NotAllowedError') {
    // 处理权限被拒绝
  } else if (error.name === 'NotFoundError') {
    // 处理未找到设备
  } else {
    // 处理其他错误
  }
}

3. 多参与者扩展

对于有许多参与者的房间,请考虑实施选择性转发单元 (SFU) 架构而不是网状网络,因为当参与者超过 4-5 人时,网状网络会变得效率低下。

安全注意事项

将应用程序部署到生产环境时,请考虑以下安全措施:

  1. 身份验证:实施适当的用户身份验证来控制房间的访问
  2. 加密:确保所有信令数据通过 HTTPS 传输
  3. 房间控制:实施房间密码或唯一标识符
  4. 速率限制:通过在服务器上实施速率限制来防止滥用
  5. 数据验证:始终验证来自客户端的所有传入数据

扩展应用程序

以下是一些增强视频聊天应用程序的想法:

  1. 屏幕共享:添加使用 getDisplayMedia()API共享屏幕的功能
  2. 聊天功能:使用 WebRTC 数据通道实现文本聊天
  3. 录音:添加会议录音功能
  4. 背景模糊/虚拟背景:实现 WebGL 或 TensorFlow.js 实现视频效果
  5. 带宽自适应:根据网络状况动态调整视频质量

结论

本文使用 WebRTC、React 和 NestJS 构建了一个完整的视频聊天应用程序。WebRTC 是一项强大的技术,可直接在浏览器中实现实时通信。结合 React 和 NestJS 等现代框架,它使开发人员无需依赖第三方服务即可创建复杂的通信应用程序。

在继续开发应用程序时,请记住正确的错误处理、安全考虑和用户体验的重要性,以创建强大且用户友好的视频聊天应用程序。

作者:ThamizhElango Natarajan

本文来自作者投稿,版权归原作者所有。如需转载,请注明出处:https://www.nxrte.com/jishu/webrtc/57790.html

(0)

相关推荐

发表回复

登录后才能评论