Northstar-dedicated - Docker container for the Northstar dedicated server. Also includes general notes on running the dedi on Linux. WIP.

Last update: Jan 6, 2023

Overview

northstar-dedicated

Docker image for the Northstar dedicated server. Not ready yet (it'll probably be another day or two).

Versioning

Tentative.

Stable

These tags will be used for stable versions from the master branch. The major version will be incremented when a breaking change is made.

{major-version}.{YYMMDD}.git{short-hash}-tf{titanfall-ver}-ns{northstar-ver} (unique tag)
{major-version}-tf{titanfall-ver}-ns{northstar-ver} (recommended if you're using custom config)
{major-version}-tf{titanfall-ver} (recommended if you're only using the documented env vars)

Note: There isn't a latest tag since the Titanfall 2 version may need to be updated manually.

Development

These tags will be used for the dev branch and for all CI builds.

dev.{YYMMDD}.git{short-hash} (unique tag)
dev (only for the main dev branch)

Titanfall 2

To use this docker container, you will need a copy of the Titanfall 2 game files to mount as a volume. You can reduce the size to around 5 gigabytes with the instructions in the next section.

Reducing the size

To slim down a Titanfall 2 install, delete the files as specified below.

delete vpk/client_sp_* englishclient_sp_*
delete r2/maps/sp_*
delete r2/paks/ except for {common,common_mp,highlight,particle_scripts}{,\(*}.rpak ui{,\(*}.dll
delete r2/sound
delete r2/media
delete r2/screenshots
on wine without origin installed:
- delete Core
- delete Support
- delete __Installer

System requirements

TODO

Running with wine

This is not relevant if you are here for the Docker container. If you want to try running this without Docker, my stubs, or my custom Wine build, I've successfully used the following configurations. Note that you will need a full X server running (Xvfb won't do).

Common

Common
Fedora 35 ext4 filesystem Wine, either: Wine 7.0.0-rc2 (distro packages) Wine 7.0.0-rc2 (source) dnf install kernel-devel mingw64-gcc gnutls-devel libjpeg-turbo-devel libpng-devel mesa-libGL-devel libunwind-devel xorg-x11-server-devel mesa-libOSMesa-devel mkdir wine-build wine-pkg pushd wine-build ../wine/configure --enable-win64 --without-alsa --without-capi --without-coreaudio --without-cups --without-dbus --without-fontconfig --without-freetype --without-gettext --without-gphoto --without-gssapi --without-gstreamer --without-netapi --without-krb5 --without-ldap --without-openal --without-opencl --without-oss --without-pcap --without-pulse --without-sane --without-sdl --without-udev --without-usb --without-v4l2 --without-vkd3d --without-vulkan --without-xcomposite --without-xcursor --without-xfixes --without-xinput --without-xinput2 --without-xrandr --without-xinerama --without-xshape --without-xshm --without-xxf86vm --with-x make -j4 popd make -C wine-build install DESTDIR=$PWD/wine-pkg X11 (so DirectX context creation doesn't fail) (Xvfb won't usually work for a few reasons)/li> Titanfall 2 (just copied game files, no need to install) Northstar 1.1.2 Mesa 21.3.2 (distro packages) `WINEARCH=win64` `WINEDEBUG=-all`
Either:
libGL (distro packages) WineD3D (built-in) `LIBGL_ALWAYS_SOFTWARE=1` `GALLIUM_DRIVER=llvmpipe`	Vulkan (distro packages) DXVK 1.9.2 `VULKAN_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json` Note: DXVK will always use your physical adapter instead of lavapipe if you have one present unless you patch this check.

Fedora 35
ext4 filesystem

Wine, either:

Wine 7.0.0-rc2 (distro packages)

Wine 7.0.0-rc2 (source)

dnf install kernel-devel mingw64-gcc gnutls-devel libjpeg-turbo-devel libpng-devel mesa-libGL-devel libunwind-devel xorg-x11-server-devel mesa-libOSMesa-devel
mkdir wine-build wine-pkg
pushd wine-build
../wine/configure --enable-win64 --without-alsa --without-capi --without-coreaudio --without-cups --without-dbus --without-fontconfig --without-freetype --without-gettext --without-gphoto --without-gssapi --without-gstreamer --without-netapi --without-krb5 --without-ldap --without-openal --without-opencl --without-oss --without-pcap --without-pulse --without-sane --without-sdl --without-udev --without-usb --without-v4l2 --without-vkd3d --without-vulkan --without-xcomposite --without-xcursor --without-xfixes --without-xinput --without-xinput2 --without-xrandr --without-xinerama --without-xshape --without-xshm --without-xxf86vm --with-x
make -j4
popd
make -C wine-build install DESTDIR=$PWD/wine-pkg

X11 (so DirectX context creation doesn't fail) (Xvfb won't usually work for a few reasons)/li>
Titanfall 2 (just copied game files, no need to install)
Northstar 1.1.2
Mesa 21.3.2 (distro packages)
WINEARCH=win64
WINEDEBUG=-all

Either:

libGL (distro packages)
WineD3D (built-in)
LIBGL_ALWAYS_SOFTWARE=1
GALLIUM_DRIVER=llvmpipe

Vulkan (distro packages)
DXVK 1.9.2
VULKAN_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.x86_64.json

Note: DXVK will always use your physical adapter instead of lavapipe if you have one present unless you patch this check.

Tracing

While working on the stubs, I needed to trace which DirectX calls Titanfall uses. You can do this by running it under WineD3D (with llvmpipe so it isn't polluted or made inconsistent by the hardware) and WINEDEBUG=trace+d3d11,trace+dxgi. Beware that this may have more than what is directly called by Titanfall due to internal implementation details of WineD3D.

TODO

Most of my progress and notes can be found on my repos and gists if anyone's interested. Some of it is also discussed on the NS discord. I've also been adding the most relevant bits to this README as I finalize them.

Note that I'm not currently accepting contributions (at least until I finish the directx stubbing and the initial version).

Wine

Figure out how to run it natively.
Figure out how to run it with wined3d + llvmpipe.
Figure out how to run it with dxvk + lavapipe.
Build minimal version.
Figure out how to run it without an X server.
Build minimal version.
Get it working on ARM64 with Box64 / Hangover / qemu-user.

Northstar

Create additional patches for optimization where necessary.
Fix console creation so it re-uses the original one.
Expose prometheus metrics?

Titanfall 2

Figure out which files are unnecessary.
Create stubs for GFSDK_{SSAO,TXAA}.win64.dll so d3d9 isn't required (thanks p0358).
Maybe: remove textures from vpks.

dxvk

Initial research and API traces.
Remove the win32 surface so it can run on xvfb.
Neutralize the render loop / presenter.
Finish removing vulkan from dxvk fork by incrementally stubbing / re-implementing APIs.
- Neutralize texture loading and shader compilation, but still return valid buffers where needed.
Remove leftover cruft.
Slim down the build system.

docker

Create initial docker image based on Fedora.
Design versioning scheme.
Create a slimmer docker image from scratch or based on Alpine with custom packages.
Figure out a way to be able to mount TF2 read-only without copying it.
Provide config via a custom entrypoint which supports env vars.
Fix Northstar logging.
GitHub actions.
Windows support? (this depends on whether my hook will work on it)
Publish image.

misc

Webpage.
Docs.

Comments

client crashes when the client connecting directly (insecure_enabled)

I am now able to replicate the issue that was happening to my docker instance when I was running it in WSL2 (In DM with @pg9182: https://discord.com/channels/@me/924866340555218974/932860684465758249)

Core issue:

Connecting via server browser works fine, while direct connect (aka connect <IP-ADDRESS>:<PORT> in console crashes the client)

Some symptoms when connecting directly, instead of master server:

(Client crashes on game start)

(Lobby menu looks different than usual + crash): https://streamable.com/r0fsgo

Logs: (Client logs) nslog2022-02-11 02-57-13.txt

(Server Logs) electric-the-phantom.log.zip

How to replicate:

Run the following on a Linux server with docker (Downloads tf2 files, and starts up the server):

#!/bin/bash
export IMAGE=ghcr.io/pg9182/northstar-dedicated:1-tf2.0.11.0-ns1.4.0
export NS_AUTH_PORT="8081"
export NS_PORT="37015"
docker pull $IMAGE

apt update -y
apt install parallel jq unzip zip -y

echo "Downloading Titanfall2 Files"

curl -L "https://ghcr.io/v2/nsres/titanfall/manifests/2.0.11.0-dedicated-mp" -s -H "Accept: application/vnd.oci.image.manifest.v1+json" -H "Authorization: Bearer QQ==" | jq -r '.layers[]|[.digest, .annotations."org.opencontainers.image.title"] | @tsv' |
{
  paths=()
  uri=()
  while read -r line; do
    while IFS=$'\t' read -r digest path; do
      path="/titanfall2/$path"
      folder=${path%/*}
      mkdir -p "$folder"
      touch "$path"
      paths+=("$path")
      uri+=("https://ghcr.io/v2/nsres/titanfall/blobs/$digest")
    done <<< "$line" ;
  done
  parallel --link --jobs 8 'wget -O {1} {2} --header="Authorization: Bearer QQ==" -nv' ::: "${paths[@]}" ::: "${uri[@]}"
}

docker run -d --pull always --publish $NS_AUTH_PORT:$NS_AUTH_PORT/tcp --publish $NS_PORT:$NS_PORT/udp --mount "type=bind,source=/titanfall2,target=/mnt/titanfall,readonly"  --env NS_SERVER_NAME="[London]electric-the-phantom" --env NS_SERVER_DESC="Competitive LTS!! Yay!" --env NS_AUTH_PORT --env NS_PORT --env NS_SERVER_PASSWORD="1306" --env NS_INSECURE="1" --name "northstar-dedicated" $IMAGE

connect via: connect <IP-ADDRESS>:37015

For convenience, can be run via Northstar-bot in lighthouse discord via: /create_server region:new jersey insecure:True, which runs the above script in vultr.

opened by L1ghtman2k 9

disable management features

I need a docker container to provide only the runtime environment to run the NorthStar server. The only other container available doesn't work at all, so I need an option for this one (or alternate version) which only runs the server and does not attempt to manage the files or configuration, as I already have that set up outside the container and the options documented in the readme are insufficient.

opened by sugoidogo 9
Poor performance on multiple different machines since 1.9.4

Not sure what's up, but I have had nothing but issues with stuttering and ping spikes since around 1.9.4. Doesn't matter what arguments I give to Northstar, the server starts to chug around 6-7 connected players. I have plenty of bandwidth, ram and CPU available. Only thing of note is I am using --network host on the docker container.

opened by pcbmaster 7

Server won't start after update to ns 1.6

After upgrade to ns 1.6 docker image dont start:, log: I have tried download again new tf2 data from EA APP, reinstall docker and even put the VM (centos7.9) from scratch. Before that, everything worked fine, no configuration changes. What did I miss?

Starting Northstar...
Starting xvfb on display :9...
002c:err:wineboot:process_run_key Error running cmd L"C:\\windows\\system32\\winemenubuilder.exe -a -r" (2).
[*] Loading stubs
[*]   Loading GFSDK_SSAO.win64.dll
[*]   Loading GFSDK_TXAA.win64.dll
[*]   Loading d3d11.dll
[*] Loading tier0.dll
[*] Loading Northstar
[2022-04-01 12:26:22.035] [info] Profile was not found in command line arguments. Using default: R2Northstar
0024:err:kerberos:kerberos_LsaApInitializePackage no Kerberos support, expect problems
0024:err:winediag:ntlm_check_version ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution.
0024:err:ntlm:ntlm_LsaApInitializePackage no NTLM support, expect problems
[12:26:22] [info] Enabling hook GetCommandLineAHook
[12:26:22] [info] Enabling hook LoadLibraryExAHook
[12:26:22] [info] Enabling hook LoadLibraryAHook
[12:26:22] [info] Enabling hook LoadLibraryExWHook
[12:26:22] [info] Enabling hook LoadLibraryWHook
[12:26:22] [info] Command line: "Z:\tmp\ns307446913\NorthstarLauncher.exe" -dedicated -multiple -port 37500 -maxplayersplaylist +setplaylistvaroverrides "run_epilogue 0 max_players 20 classic_mp 0 scorelimit 100" ns_private_match_only_host_can_change_settings 2 +setplaylist private_match
0024:fixme:msvcp:_Locinfo__Locinfo_ctor_cat_cstr (00000000007CF150 1 C) semi-stub
[12:26:22] [info] Loading resource from library
0024:err:module:import_dll Library ktmw32.dll (which is needed by L"Z:\\tmp\\ns307446913\\discord_game_sdk.dll") not found
0024:err:module:import_dll Library discord_game_sdk.dll (which is needed by L"Z:\\tmp\\ns307446913\\R2Northstar\\plugins\\DiscordRPC.dll") not found
[12:26:22] [info] Failed to load library Module not found.:
[*] Loading launcher.dll
[*] Launching the game...
[12:26:22] [info] Enabling hook ReadFileFromVPKHook
[12:26:22] [info] Enabling hook ReadFromCacheHook
[12:26:22] [info] Enabling hook AddSearchPathHook
[12:26:22] [info] Enabling hook ReadFileFromFilesystemHook
[12:26:22] [info] Enabling hook MountVPKHook
0024:fixme:heap:RtlSetHeapInformation 0000000036490000 0 00000000007CDD50 4 stub
[12:26:22] [info] InitialiseDedicated
[12:26:22] [info] Ready to receive console commands.
0134:fixme:msvcp:_Locinfo__Locinfo_ctor_cat_cstr (00000000394FFC70 1 C) semi-stub
[12:26:22] [info] Enabling hook IsGameActiveWindowHook
[12:26:22] [info] Enabling hook ConVar::IsFlagSet
[12:26:22] [info] Registering ConCommand force_newgame
[12:26:22] [info] Registering ConCommand ns_start_reauth_and_leave_to_lobby
[12:26:22] [info] Registering ConCommand ns_end_reauth_and_leave_to_lobby
[12:26:22] [info] Registering Convar spewlog_enable
[12:26:22] [info] Enabling hook EngineSpewFuncHook
[12:26:22] [info] Enabling hook Status_ConMsg_Hook
[12:26:22] [info] Enabling hook CClientState_ProcessPrint_Hook
[12:26:22] [info] Registering ConCommand ban
[12:26:22] [info] Registering ConCommand unban
[12:26:22] [info] Registering ConCommand clearbanlist
[12:26:22] [info] Registering Convar ns_erase_auth_info
[12:26:22] [info] Registering Convar ns_auth_allow_insecure
[12:26:22] [info] Registering Convar ns_auth_allow_insecure_write
[12:26:22] [info] Registering Convar sv_quota_stringcmdspersecond
[12:26:22] [info] Registering Convar net_chan_limit_mode
[12:26:22] [info] Registering Convar net_chan_limit_msec_per_sec
[12:26:22] [info] Registering Convar ns_player_auth_port
[12:26:22] [info] Registering Convar sv_querylimit_per_sec
[12:26:22] [info] Registering Convar sv_max_chat_messages_per_sec
[12:26:22] [info] Registering ConCommand ns_resetpersistence
[12:26:22] [info] Enabling hook CBaseServer__ConnectClientHook
[12:26:22] [info] Enabling hook CBaseClient__ConnectHook
[12:26:22] [info] Enabling hook CBaseClient__ActivatePlayerHook
[12:26:22] [info] Enabling hook CBaseClient__DisconnectHook
[12:26:22] [info] Enabling hook CGameClient__ExecuteStringCommandHook
[12:26:22] [info] Enabling hook CNetChan___ProcessMessagesHook
[12:26:22] [info] Enabling hook CBaseClient__SendServerInfoHook
[12:26:22] [info] Enabling hook ProcessConnectionlessPacketHook
[12:26:22] [info] Registering Convar ns_masterserver_hostname
[12:26:23] [info] Registering Convar ns_server_name
[12:26:23] [info] Registering Convar ns_server_desc
[12:26:23] [info] Registering Convar ns_server_password
[12:26:23] [info] Registering Convar ns_report_server_to_masterserver
[12:26:23] [info] Registering Convar ns_report_sp_server_to_masterserver
[12:26:23] [info] Registering ConCommand ns_fetchservers
[12:26:23] [info] Enabling hook CHostState__State_NewGameHook
[12:26:23] [info] Enabling hook CHostState__State_ChangeLevelMPHook
[12:26:23] [info] Enabling hook CHostState__State_ChangeLevelSPHook
[12:26:23] [info] Enabling hook CHostState__State_GameShutdownHook
[12:26:23] [info] Registering ConCommand setplaylist
[12:26:23] [info] Registering ConCommand setplaylistvaroverrides
[12:26:23] [info] Registering Convar ns_use_clc_SetPlaylistVarOverride
[12:26:23] [info] Enabling hook Onclc_SetPlaylistVarOverrideHook
[12:26:23] [info] Enabling hook SetPlaylistVarOverrideHook
[12:26:23] [info] Enabling hook GetCurrentPlaylistVarHook
[12:26:23] [info] Enabling hook GetCurrentGamemodeMaxPlayersHook
[12:26:23] [error] MH_CreateHook failed for function LoadPakSyncHook
[12:26:23] [info] Enabling hook LoadPakAsyncHook
[12:26:23] [info] Enabling hook KeyValues__LoadFromBufferHook
[12:26:23] [info] Loaded mod Northstar.Client successfully
[12:26:23] [info] Mod Northstar.Client is enabled
[12:26:23] [info] Loaded mod Northstar.Custom successfully
[12:26:23] [info] Mod Northstar.Custom is enabled
[12:26:23] [info] Loaded mod Northstar.CustomServers successfully
[12:26:23] [info] Mod Northstar.CustomServers is enabled
[12:26:23] [info] Registering Convar filter_hide_empty
[12:26:23] [info] Registering Convar filter_hide_full
[12:26:23] [info] Registering Convar filter_hide_protected
[12:26:23] [info] Registering Convar filter_map
[12:26:23] [info] Registering Convar filter_gamemode
[12:26:23] [info] Registering Convar filter_mods
[12:26:23] [info] Registering Convar filter_map_hide_locked
[12:26:23] [info] Registering Convar ns_lobby_type
[12:26:23] [info] Registering Convar ns_is_modded_server
[12:26:23] [info] Registering Convar ns_should_return_to_lobby
[12:26:23] [info] Registering Convar ns_allow_spectators
[12:26:23] [info] Registering Convar ns_private_match_last_mode
[12:26:23] [info] Registering Convar ns_private_match_last_map
[12:26:23] [info] Registering Convar ns_private_match_only_host_can_change_settings
[12:26:23] [info] Registering Convar ns_private_match_only_host_can_start
[12:26:23] [info] Registering Convar ns_private_match_countdown_length
[12:26:23] [info] Registering Convar ns_private_match_override_maxplayers
[12:26:23] [info] Registering Convar ns_should_log_unknown_clientcommands
[12:26:23] [info] Registering Convar ns_disallowed_tacticals
[12:26:23] [info] Registering Convar ns_disallowed_tactical_replacement
[12:26:23] [info] Registering Convar ns_disallowed_weapons
[12:26:23] [info] Registering Convar ns_disallowed_weapon_primary_replacement
[12:26:23] [info] Registering Convar ns_force_melee
[12:26:23] [info] Registering ConCommand reload_mods
[12:26:23] [info] Enabling hook D3D11CreateDeviceHook
d3d11: D3D11CreateDevice: initializing d3d11 stub for northstar (github.com/R2Northstar/NorthstarStubs)
d3d11: stub: GetDesc
0024:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
[12:26:23] [info] MountVPK vpk/client_frontend.bsp
0024:fixme:keyboard:LoadKeyboardLayoutW name L"00000409", flags 0, semi-stub!
0140:fixme:service:I_ScRegisterDeviceNotification Notification filters are not yet implemented.
00fc:fixme:imm:ImeSetActiveContext (0x255f40, 0): stub
00fc:fixme:imm:ImmReleaseContext (0000000000010020, 0000000000255F40): stub
013c:fixme:imm:ImeSetActiveContext (0x3be7b5a0, 1): stub
013c:fixme:imm:ImmReleaseContext (0000000000020068, 000000003BE7B5A0): stub
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
warning: Unknown nb_ctl request:  4
Error: Failed to run server: server exited: exit status 18.

opened by we-will-all-die 6

Sever server lag introduced by patches in the last few days

One of the commits between: 9febe7ee772fd35b636421a421e9f506af83013b and 9fb610ea9a3ea94584525f3ec013c8c1b90b6a84 introduces severe server-side lag: https://streamable.com/8cl7x7

opened by L1ghtman2k 4
permission denied on CentOS - Podman emulator
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg. Trying to pull ghcr.io/pg9182/northstar-dedicated:1-tf2.0.11.0... Getting image source signatures Copying blob a99c34865d99 skipped: already exists
Copying blob 891382f67258 skipped: already exists
Copying blob 617ce86479e7 skipped: already exists
Copying blob 59bf1c3509f3 skipped: already exists
Copying blob 1d676022bd00 skipped: already exists
Copying blob aca1971a86b4 skipped: already exists
Copying blob d8e612b3ace8 skipped: already exists
Copying blob c49e47170caa skipped: already exists
Copying config 760369964c done
Writing manifest to image destination Storing signatures Northstar Dedicated Server - Docker (hostname: f7938bcd6dfe)

https://northstar.tf https://northstar.tf/discord https://github.com/R2Northstar/Northstar https://github.com/pg9182/northstar-dedicated

Merging files... Error: Failed to merge game files: merge titanfall: access "/mnt/titanfall/build.txt": stat /mnt/titanfall/build.txt: permission denied
opened by Archer1411 3

Game wont initialize

I copied the files from a install of Titanfall 2 and slimmed it down, I made sure that I didn't remove any of the file that I wasn't supposed to remove. The dedicated server is stuck on "Loading the game..." and I have no idea how to make it work, I read a reddit post saying that I have to launch the game but I cannot do that.

Log:

1-tf2.0.11.0: Pulling from pg9182/northstar-dedicated
Digest: sha256:918c8d69ce10d13d30ca87913dcf5ded36dc0226757cb56321707e6a4d940eb9
Status: Image is up to date for ghcr.io/pg9182/northstar-dedicated:1-tf2.0.11.0
Northstar Dedicated Server - Docker (hostname: f29b2e1539d3)

    https://northstar.tf
    https://northstar.tf/discord
    https://github.com/R2Northstar/Northstar
    https://github.com/pg9182/northstar-dedicated

Merging files...

    Bind mounts:
        /mnt/titanfall ext4 ro,relatime,discard

Merging configuration...

    Extra arguments:
        -port
        37015
    Config:
        +base_tickinterval_mp "0.016666667" (from default)
        +everything_unlocked "1" (from default)
        +host_skip_client_dll_crc "1" (from default)
        +net_chan_limit_mode "2" (from default)
        +net_chan_limit_msec_per_sec "100" (from default)
        +net_data_block_enabled "0" (from default)
        +ns_auth_allow_insecure "0" (from default) (NS_INSECURE)
        +ns_erase_auth_info "1" (from default)
        +ns_masterserver_hostname "https://northstar.tf" (from default) (NS_MASTERSERVER_URL)
        +ns_player_auth_port "8081" (from default) (NS_PORT_AUTH)
        +ns_report_server_to_masterserver "1" (from default) (NS_MASTERSERVER_REGISTER)
        +ns_report_sp_server_to_masterserver "0" (from default) (NS_MASTERSERVER_REGISTER)
        +ns_server_desc "" (from env:NS_SERVER_DESC) (NS_SERVER_DESC)
        +ns_server_name "[AUS] Sam" (from env:NS_SERVER_NAME) (NS_SERVER_NAME)
        +ns_server_password "" (from default) (NS_SERVER_PASSWORD)
        +ns_should_return_to_lobby "1" (from default)
        +sv_max_snapshots_multiplayer "300" (from default)
        +sv_minupdaterate "20" (from default)
        +sv_querylimit_per_sec "15" (from default)
        +sv_updaterate_mp "20" (from default)

Starting Northstar...
nswrap: a78158a513220f6529cbab613f8ffcc88b8714f6b57895b10462240ec2f2f494
nswrap:
nswrap: config
nswrap:   PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
nswrap:   HOME=/home/northstar
nswrap:   USER=(null)
nswrap:   HOSTNAME=f29b2e1539d3
nswrap:   DISPLAY=xvfb
nswrap:   WINEPREFIX=/home/northstar/.wine
nswrap:   WINEDEBUG=(null)
nswrap:   WINESERVER=(null)
nswrap:
nswrap: system info:
nswrap:   kernel: Linux f29b2e1539d3 5.13.0-1023-azure #27~20.04.1-Ubuntu SMP Mon Apr 25 22:39:07 UTC 2022 x86_64
nswrap:   processor: 2 cores
nswrap:   memory: 8340144128 total, 1530220544 free, 1097728 shared, 28057600 buffer
nswrap:   swap: 0 total, 0 free
nswrap:
nswrap: using WINEDEBUG='fixme-secur32,fixme-bcrypt,fixme-ver,err-wldap32'
nswrap: warning: currently, at least 3 cores are required, but only 2 were found
nswrap: starting xvfb
nswrap: xvfb started on display :0 with pid 14
nswrap: starting wine
002c:err:wineboot:process_run_key Error running cmd L"C:\\windows\\system32\\winemenubuilder.exe -a -r" (2).
[*] Loading stubs
[*]   Loading GFSDK_SSAO.win64.dll
[*]   Loading GFSDK_TXAA.win64.dll
[*]   Loading d3d11.dll
[*] Loading tier0.dll
[*] Loading Northstar
[2022-05-15 02:12:47.937] [info] Profile was not found in command line arguments. Using default: R2Northstar
0024:err:kerberos:kerberos_LsaApInitializePackage no Kerberos support, expect problems
0024:err:winediag:ntlm_check_version ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution.
0024:err:ntlm:ntlm_LsaApInitializePackage no NTLM support, expect problems
[02:12:47] [info] Enabling hook GetCommandLineAHook
[02:12:47] [info] Enabling hook LoadLibraryExAHook
[02:12:47] [info] Enabling hook LoadLibraryAHook
[02:12:47] [info] Enabling hook LoadLibraryExWHook
[02:12:47] [info] Enabling hook LoadLibraryWHook
[02:12:47] [info] Command line: "Z:\tmp\ns2258892549\NorthstarLauncher.exe" -dedicated -port 37015 +setplaylist private_match
0024:fixme:msvcp:_Locinfo__Locinfo_ctor_cat_cstr (00000000007CF100 1 C) semi-stub
[02:12:47] [info] NorthstarLauncher version: 1.7.0.0
[02:12:48] [warning] Could not find any plugins. Skipped loading plugins
[*] Loading launcher.dll
[*] Launching the game...
nswrap: warning: watchdog: watchdog did not receive enough title updates for initialization: only received 0/10 initial ticks less than 60s apart within 240s

opened by blockbuster206 2

Servers intermittently failing to start

Logs from instance finer-dormammu (457ea7c023e6):

--- snip ---
2022-05-11T22:05:54.485600351Z  nswrap: 3db9f64ef7797935daa47aba87436a10c7ce41d6a83ac378c364e22b2dc1ffc0
2022-05-11T22:05:54.485627286Z  nswrap: 
2022-05-11T22:05:54.485631616Z  nswrap: config
2022-05-11T22:05:54.485635274Z  nswrap:   PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
2022-05-11T22:05:54.485638842Z  nswrap:   HOME=/home/northstar
2022-05-11T22:05:54.485642319Z  nswrap:   USER=(null)
2022-05-11T22:05:54.485645757Z  nswrap:   HOSTNAME=457ea7c023e6
2022-05-11T22:05:54.485649464Z  nswrap:   DISPLAY=xvfb
2022-05-11T22:05:54.485653175Z  nswrap:   WINEPREFIX=/home/northstar/.wine
2022-05-11T22:05:54.485656906Z  nswrap:   WINEDEBUG=(null)
2022-05-11T22:05:54.485660459Z  nswrap:   WINESERVER=(null)
2022-05-11T22:05:54.485664104Z  nswrap: 
2022-05-11T22:05:54.485667546Z  nswrap: system info:
2022-05-11T22:05:54.485671227Z  nswrap:   kernel: Linux 457ea7c023e6 5.4.0-84-generic #94-Ubuntu SMP Thu Aug 26 20:27:37 UTC 2021 x86_64
2022-05-11T22:05:54.485675010Z  nswrap:   processor: 4 cores
2022-05-11T22:05:54.485678715Z  nswrap:   memory: 8348397568 total, 4034752512 free, 2465792 shared, 30863360 buffer
2022-05-11T22:05:54.485682327Z  nswrap:   swap: 0 total, 0 free
2022-05-11T22:05:54.485686022Z  nswrap: 
2022-05-11T22:05:54.485859802Z  nswrap: using WINEDEBUG='fixme-secur32,fixme-bcrypt,fixme-ver,err-wldap32'
2022-05-11T22:05:54.485869053Z  nswrap: starting xvfb
2022-05-11T22:05:54.546217460Z  nswrap: xvfb started on display :0 with pid 16
2022-05-11T22:05:54.546255533Z  nswrap: starting wine
2022-05-11T22:05:54.546259554Z  nswrap: warning: xvfb terminated: exited with status 1
2022-05-11T22:05:54.546227555Z  (EE) 
2022-05-11T22:05:54.546295759Z  Fatal server error:
2022-05-11T22:05:54.546299809Z  (EE) Cannot write display number to fd 15
2022-05-11T22:05:54.546302400Z  (EE) 
2022-05-11T22:05:54.661885133Z  002c:err:wineboot:process_run_key Error running cmd L"C:\\windows\\system32\\winemenubuilder.exe -a -r" (2).
2022-05-11T22:05:55.588632860Z  [*] Loading stubs
2022-05-11T22:05:55.591654056Z  [*]   Loading GFSDK_SSAO.win64.dll
2022-05-11T22:05:55.594523452Z  [*]   Loading GFSDK_TXAA.win64.dll
2022-05-11T22:05:55.597491307Z  [*]   Loading d3d11.dll
2022-05-11T22:05:55.603497809Z  [*] Loading tier0.dll
2022-05-11T22:05:55.675831768Z  [*] Loading Northstar
2022-05-11T22:05:55.749746925Z  [2022-05-11 22:05:55.718] [[32minfo[m] Profile was not found in command line arguments. Using default: R2Northstar
2022-05-11T22:05:55.760466891Z  0024:err:kerberos:kerberos_LsaApInitializePackage no Kerberos support, expect problems
2022-05-11T22:05:55.789040898Z  0024:err:winediag:ntlm_check_version ntlm_auth was not found or is outdated. Make sure that ntlm_auth >= 3.0.25 is in your path. Usually, you can find it in the winbind package of your distribution.
2022-05-11T22:05:55.795982524Z  0024:err:ntlm:ntlm_LsaApInitializePackage no NTLM support, expect problems
2022-05-11T22:05:55.814486202Z  [22:05:55] [info] Enabling hook GetCommandLineAHook
2022-05-11T22:05:55.831579948Z  [22:05:55] [info] Enabling hook LoadLibraryExAHook
2022-05-11T22:05:55.843374783Z  [22:05:55] [info] Enabling hook LoadLibraryAHook
2022-05-11T22:05:55.857997772Z  [22:05:55] [info] Enabling hook LoadLibraryExWHook
2022-05-11T22:05:55.873518540Z  [22:05:55] [info] Enabling hook LoadLibraryWHook
2022-05-11T22:05:55.899822143Z  [22:05:55] [info] Command line: "Z:\tmp\ns2660680541\NorthstarLauncher.exe" -dedicated -port 37015 +setplaylist private_match
2022-05-11T22:05:55.906329919Z  0024:fixme:msvcp:_Locinfo__Locinfo_ctor_cat_cstr (00000000007CF100 1 C) semi-stub
2022-05-11T22:05:55.908433477Z  [22:05:55] [info] NorthstarLauncher version: 1.7.0.0
--- snip ---
2022-05-11T22:05:57.528425232Z  [22:05:57] [info] Enabling hook D3D11CreateDeviceHook
2022-05-11T22:05:58.001662738Z  d3d11: D3D11CreateDevice: initializing d3d11 stub for northstar (github.com/R2Northstar/NorthstarStubs)
2022-05-11T22:05:58.006131020Z  0024:fixme:ntdll:NtQuerySystemInformation info_class SYSTEM_PERFORMANCE_INFORMATION
2022-05-11T22:05:58.021129823Z  [22:05:58] [info] MountVPK vpk/client_frontend.bsp
2022-05-11T22:05:58.137268477Z  0024:fixme:keyboard:LoadKeyboardLayoutW name L"00000409", flags 0, semi-stub!
2022-05-11T22:05:58.235654189Z  0118:err:winediag:nodrv_CreateWindow Application tried to create a window, but no driver could be loaded.
2022-05-11T22:05:58.235705862Z  0118:err:winediag:nodrv_CreateWindow L"Make sure that your X server is running and that $DISPLAY is set correctly."
2022-05-11T22:05:58.251625650Z  nswrap: northstar exited with status 1
2022-05-11T22:05:58.251677207Z  nswrap: waiting for children to exit

Recently, this has been occurring more frequently.

opened by pg9182 2

Snyk container vulnerability scan results

just thought i would put this here in case you're interested

Testing ghcr.io/pg9182/northstar-dedicated:1-tf2.0.11.0...

✗ Low severity vulnerability found in busybox/busybox
  Description: ALPINE-13661
  Info: https://snyk.io/vuln/SNYK-ALPINE315-BUSYBOX-2606932
  Introduced through: busybox/[email protected], alpine-baselayout/[email protected], ca-certificates/ca-certificates@20211220-r0, fontconfig/[email protected], mkfontscale/[email protected], busybox/[email protected]
  From: busybox/[email protected]
  From: alpine-baselayout/[email protected] > busybox/[email protected]
  From: ca-certificates/ca-certificates@20211220-r0 > busybox/[email protected]
  and 3 more...
  Image layer: 'apk add --no-cache gnutls tzdata ca-certificates sudo'
  Fixed in: 1.34.1-r5

✗ Medium severity vulnerability found in util-linux/libuuid
  Description: Information Exposure
  Info: https://snyk.io/vuln/SNYK-ALPINE315-UTILLINUX-2401805
  Introduced through: util-linux/[email protected], fontconfig/[email protected], libsm/[email protected], util-linux/[email protected], xinit/[email protected]
  From: util-linux/[email protected]
  From: fontconfig/[email protected] > util-linux/[email protected]
  From: libsm/[email protected] > util-linux/[email protected]
  and 2 more...
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.37.4-r0

✗ Medium severity vulnerability found in expat/expat
  Description: Resource Exhaustion
  Info: https://snyk.io/vuln/SNYK-ALPINE315-EXPAT-2407746
  Introduced through: expat/[email protected], fontconfig/[email protected], mesa/[email protected]
  From: expat/[email protected]
  From: fontconfig/[email protected] > expat/[email protected]
  From: mesa/[email protected] > expat/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.4.5-r0

✗ High severity vulnerability found in zlib/zlib
  Description: Out-of-bounds Write
  Info: https://snyk.io/vuln/SNYK-ALPINE315-ZLIB-2434420
  Introduced through: zlib/[email protected], apk-tools/[email protected], mkfontscale/[email protected], freetype/[email protected], libpng/[email protected], libfontenc/[email protected], libxfont2/[email protected], sudo/[email protected]_p2-r1
  From: zlib/[email protected]
  From: apk-tools/[email protected] > zlib/[email protected]
  From: mkfontscale/[email protected] > zlib/[email protected]
  and 5 more...
  Image layer: 'apk add --no-cache gnutls tzdata ca-certificates sudo'
  Fixed in: 1.2.12-r0

✗ High severity vulnerability found in openssl/libcrypto1.1
  Description: Loop with Unreachable Exit Condition ('Infinite Loop')
  Info: https://snyk.io/vuln/SNYK-ALPINE315-OPENSSL-2426331
  Introduced through: openssl/[email protected], openssl/[email protected], apk-tools/[email protected], libretls/[email protected], ca-certificates/ca-certificates@20211220-r0
  From: openssl/[email protected]
  From: openssl/[email protected] > openssl/[email protected]
  From: apk-tools/[email protected] > openssl/[email protected]
  and 5 more...
  Image layer: 'apk add --no-cache gnutls tzdata ca-certificates sudo'
  Fixed in: 1.1.1n-r0

✗ High severity vulnerability found in libretls/libretls
  Description: Loop with Unreachable Exit Condition ('Infinite Loop')
  Info: https://snyk.io/vuln/SNYK-ALPINE315-LIBRETLS-2428776
  Introduced through: libretls/[email protected], busybox/[email protected]
  From: libretls/[email protected]
  From: busybox/[email protected] > libretls/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 3.3.4-r3

✗ High severity vulnerability found in expat/expat
  Description: Integer Overflow or Wraparound
  Info: https://snyk.io/vuln/SNYK-ALPINE315-EXPAT-2407747
  Introduced through: expat/[email protected], fontconfig/[email protected], mesa/[email protected]
  From: expat/[email protected]
  From: fontconfig/[email protected] > expat/[email protected]
  From: mesa/[email protected] > expat/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.4.5-r0

✗ Critical severity vulnerability found in expat/expat
  Description: Integer Overflow or Wraparound
  Info: https://snyk.io/vuln/SNYK-ALPINE315-EXPAT-2407738
  Introduced through: expat/[email protected], fontconfig/[email protected], mesa/[email protected]
  From: expat/[email protected]
  From: fontconfig/[email protected] > expat/[email protected]
  From: mesa/[email protected] > expat/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.4.5-r0

✗ Critical severity vulnerability found in expat/expat
  Description: Improper Encoding or Escaping of Output
  Info: https://snyk.io/vuln/SNYK-ALPINE315-EXPAT-2407749
  Introduced through: expat/[email protected], fontconfig/[email protected], mesa/[email protected]
  From: expat/[email protected]
  From: fontconfig/[email protected] > expat/[email protected]
  From: mesa/[email protected] > expat/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.4.5-r0

✗ Critical severity vulnerability found in expat/expat
  Description: Exposure of Resource to Wrong Sphere
  Info: https://snyk.io/vuln/SNYK-ALPINE315-EXPAT-2407753
  Introduced through: expat/[email protected], fontconfig/[email protected], mesa/[email protected]
  From: expat/[email protected]
  From: fontconfig/[email protected] > expat/[email protected]
  From: mesa/[email protected] > expat/[email protected]
  Image layer: Introduced by your base image (alpine:3.15.0)
  Fixed in: 2.4.5-r0

✗ Critical severity vulnerability found in busybox/busybox
  Description: CVE-2022-28391
  Info: https://snyk.io/vuln/SNYK-ALPINE315-BUSYBOX-2440607
  Introduced through: busybox/[email protected], alpine-baselayout/[email protected], ca-certificates/ca-certificates@20211220-r0, fontconfig/[email protected], mkfontscale/[email protected], busybox/[email protected]
  From: busybox/[email protected]
  From: alpine-baselayout/[email protected] > busybox/[email protected]
  From: ca-certificates/ca-certificates@20211220-r0 > busybox/[email protected]
  and 3 more...
  Image layer: 'apk add --no-cache gnutls tzdata ca-certificates sudo'
  Fixed in: 1.34.1-r5

opened by laundmo 2

Fix entrypoint hanging when launcher crashes

These changes allow the entrypoint to terminate if the Northstar launcher crashes, allowing proper auto-restart functionality with docker-compose. Since sendTerminate(true) is deferred to the end of n.Run(), extra calls within Run() are not required. Also, initialising Xrdb now removes old lock files, which in my case were preventing the docker container from restarting successfully.

opened by mitchpk 2
Update Northstar to 1.4, adjust instance creation
Sorry about the PR spam, not sure if you have something like this cooking already but this is hopefully at least useful for others using this image.

Updates the Northstar build file to use 1.4

Adjusts NSInstance logic to only abort when NorthstarLauncher exits instead of creating a watcher process, which isn't necessary anymore

Fixes what seems to be a deadlock when the launcher crashes, stopping the entrypoint from exiting (I'm not super sure if my fix is correct, but it does work)

I haven't added support for running NS commands with stdin, not totally sure how that would be set up with the custom PTY stuff?

Also apologies in advance, first time using Go 😄
opened by cpdt 2
Rework build/update/release process
Goals:

Support multiple northstar releases at HEAD.

Support getting northstar from CI artifacts in addition to releases.

Automate more of the update process, and support doing it before a full release.

Support selectively updating the image for past releases.

Proper staging environment (with promotions and rollbacks of specific versions) instead of just the dev build -> approval -> release process.

Allow more people to approve updates.

Should not change anything about the built image; it'll still be 100% compatible with what is currently done including the /usr/lib/northstar dir that some people depend on (even though it's technically an unsupported implementation detail).

Tasks:

[ ] Split merge stuff, support configuration for each ns version.

[ ] Decide whether to push all old versions too by default or only when triggered manually (probably the latter)

[ ] If the latter, write a script to re-tag the latest version to keep the order and recommended install command in gh consistent.

[ ] Maybe split config stuff too.

[ ] Decide whether to make the Dockerfile templated or whether to add a build arg.

[ ] Add northstar version manifest for keeping track of known (to be compatible), supported (whether to support building for it, this will mostly be keeping stuff as long as possible), and updated (whether to allow pushing new images) versions.

[ ] Decide on a policy (and possibly automation of it) for which versions are going to be updated and supported.

[ ] Implement pipelines

[ ] Dev build northstar-dedicated-dev (probably all supported versions, or maybe just the latest with a manually triggered one for all supported versions).

[ ] Staging release northstar-dedicated-stg:1... (either manually triggered or based on tags, either build from scratch or push the corresponding dev build if it's the same).

[ ] Release northstar-dedicated:1... (promoted from staging) (also look into having this vote-based with more people allowed to vote) (also look into integrating this with l1ghtman's server management bot)

[ ] Maybe one for opening a PR for the latest NS release.

[ ] Maybe one for dropping old versions.

[ ] Investigate the possibility of removing package metadata from the layers and/or using buildkit's linked copy to make layers completely independent and deduplicatable.

[ ] #20

[ ] Document the update/release process, including steps to test images.

[ ] Add additional repo collaborators.

[ ] Maybe move to the NS org (or a new one) for more granular permissions.

[ ] Add proper issue tags.
opened by pg9182 0