From The Mana World
m (Fix a typo)
m (Move page script moved page Dev:How to Develop to Development:How to Develop: Rename Dev: to Development: so the miraheze import works)
 
(137 intermediate revisions by 22 users not shown)
Line 1: Line 1:
=Set Up=
==Developing for The Mana World==
This is a how to help content development for official TMW-eA using Ubuntu 10.04 on amd64 and the Mana client.  Other OS and clients may differ.  Following this guide, you should be able to get development testing versions of the client data (tmwdata), server-data (eathena-data) and participate in development by using a localhost server.  After getting these, you can both test current development and begin development in the direction you find most fitting.


'''Dependencies:''' Throughout this tutorial, I do not address dependencies.  Bison and Flex are needed, as are other dependencies.  This will be inserted with time.
Note: The links here are for the rEvolt Project.


==Set Up Own Server==
For TMW Classic, you should replace <code><nowiki>gitlab.com/evol</nowiki></code> with <code><nowiki>git.themanaworld.org/evolved</nowiki></code>, keeping other parts intact.


=== Quick Start for local server ===


'''Here are the steps to use when creating a new local testing server.'''
* Open a terminal.
* ''mkdir ~/tmw-ea''
* ''cd ~/tmw-ea/''
* ''git clone git://gitorious.org/tmw-eathena/mainline.git eathena''
* ''git clone --recursive git://gitorious.org/+tmw-admins/tmw/tmwa-server-test.git eathena-data''
* ''cd eathena/''
* ''make''
* ''sudo cp login-server char-server map-server ladmin /usr/local/bin/''
* ''git config --global url.git@gitorious.org:.pushInsteadOf git://gitorious.org'' {ref|push} {ref|once}
* ''cd ~/tmw-ea/eathena-data''
* ''make conf''
* ''cd client-data''
* ''git checkout master''


==== Install Dependencies ====


'''You can now launch your test server.'''
Depending on what sql and OS your using install Maria or Mysql packages:
* Open a terminal.
* Maria packages: mariadb-server, mariadb-client, mariadbclient-dev, libmariadbclient-dev
* ''cd ~/tmw-ea/eathena-data/''
* Mysql packages: mysqldb-server, mysqldb-client, mysqlclient-dev, libmysqlclient-dev
* ''./char-server& ./login-server& ./map-server&''
* Open another terminal.
* ''mana --server localhost --port 6901 -u -d ~/tmw-ea/eathena-data/client-data''
This should run all three of your servers and the client.


All these other dependencies need to be installed regardless of sql used:
* zlib-dev or zlib1g-dev package, libpcre3-dev, gcc, make, automake, autoconf, libtool


'''Make yourself a GM so you can test changes made.'''
Some distributions also require manual installation of ssl and crypto packages:
See [[#Administration]]
* libssl-dev, libcrypto++-dev


'''See also:''' [[GM Commands]]
Note: Some packages may named differently depending on your distro.<br />
You may use <code>apt-cache search libpcre</code> to check all packages which contain 'libpcre' on the name, and hopefully find the right one.
It may be named <code>libpcre0v5-dev</code> for example.
You may try with <code>libpcre++-dev</code> first.


apt-cache is not available on non-Debian or non-Debian-based distros.<br />
You may want to use <code>apt search <package(s)></code> instead.


'''You are now free to do some changes, first close the client and your servers.'''
You need gcc <= 9 to compile the servers. gcc-10 and later will raise errors when compiling md5 functions. We advise to install <code>gcc-9</code> packages and pass it under CC environment variable.


'''''Notes for client data changes: 1) commit and push them in the client-data repo 2) commit and push "client-data" when it shows "(new commits)" in your server-data 3) commit and push other server data changes.'''''
==== Open command line/terminal ====
* Make your modifications (e.g. Change a map) and save them.
* Open a terminal.
* ''cd ~/tmw-ea/eathena-data/client-data''
* ''git status''
* ''git add [modified files]''
* ''git commit''
* ''git push'' {{note|push}}
* ''cd ..''
* ''make maps'' (if you changed a map)
* ''git status''
* ''git add [modified files]''
* ''git add client-data'' (if ''git status'' shows ''(new commits)''. '''Don't do this if you don't have push access to client-data!''')
* ''git commit''
* ''git push'' {{note|push}}
* Close the terminal.


{{note|push}} : a step for those with commit access to the gitorious repositories.  For others, you don't need to reconfigure your clone and please make a patch on steps where ''git push'' is suggested.
<code> ~$</code>
{{note|once}} : you only need to do this once, no matter how many times you independently clone the repositories.


*''git format-patch -M -C HEAD^''
==== Clone repository ====
* pastebin the patch and post for review at irc.themanaworld.org channel: #tmw-dev or on our [http://forums.themanaworld.org/viewforum.php?f=18 development forums].


run command:


'''You can also pull the data which has been modified.'''
<code>git clone https://git.themanaworld.org/revolt/all.git</code>
* Open a terminal.
* ''cd ~/tmw-ea/eathena-data''
* ''git pull''
* ''git submodule update --merge''
* Close the terminal.


'''See Also:''' [http://wiki.themanaworld.org/images/3/33/README.mv.txt README.mv.txt] - Documentation on using git submodules, like client-data and tmw-music.
You will see something like this:


== Administration ==
<code><pre>$ git clone https://git.themanaworld.org/revolt/all.git
# Start the servers.
Cloning into 'evol-all'...
# Open a command line terminal.
remote: Counting objects: 69, done.
# cd ~/tmw-ea/eathena-data/login
remote: Compressing objects: 100% (67/67), done.
# ladmin (not ./ladmin)
remote: Total 69 (delta 36), reused 0 (delta 0)
# add username M password
Receiving objects: 100% (69/69), 7.05 KiB | 0 bytes/s, done.
# gm username 99
Resolving deltas: 100% (36/36), done.
Checking connectivity... done.</pre></code>


== Connecting ==
'''Note: For TMW Classic, you should run this command instead:'''
# Open a command line terminal.
<code>git clone https://git.themanaworld.org/evolved/all.git</code>
# Run your client pointing to your client data and your local server. E.g. mana -u -d ~/tmw-ea/tmwdata --server localhost --port 6901


If all goes well, you are now ready to test new content and participate with development.
'''Everything else should be roughly the same afterward.'''


==== Init repos ====


In evol-all directory run command
: <code>./init.sh all</code>


=Making Content=
==== Build and set-up local server ====
==Making Map Content==
===Java Converter Tool===
This has been my experience on running the java Converter tools, which takes map data from client-data and applies it to the server-data.  The README addresses dependencies for using this tool.  It suggests there are other ways it can work in the README, if this method does work for you.


Edit your map and save it. You can push the client data now. Now, you need to run the converter.
In directory '''client-data''' in evol-all run the following command:
# cd ~/tmw-ea/eathena-data/
# make maps
After that, commit and push the files that the converter edited.


==Getting Art Content Into The Game==
: <code>ln -s ../music/music music</code>
 
If you want to be able to listen to background music.
 
 
In directory '''server-data''' in evol-all run the following command:
 
: <code>make new</code>
 
On multi-core/thread CPUs try appending -j<number of cores> to delegate a task to every core/thread like <code>make new -j4</code>
 
(Note: Mariadb/MySQL needs to be up and running for this command to execute correctly, Debian/Ubuntu may automatically do this setup on installation. Read your distro's manuals for SQL setup and configuration.)
 
==== Running server ====
 
You need to repeat this step every time your computer shuts down,
or every time you close your terminal.
 
First, you should be in server-data folder.
 
In three separate console windows, run the following commands:
 
: <code>./login-server</code>
: <code>./char-server</code>
: <code>./map-server</code>
 
Pay attention to any bug reported in any of them, as they often mean
something went wrong in the previous steps.
 
Keep in mind that if you close the terminal windows, you'll need to
run these commands again. You should preferably run them in this order.
 
Note: You may also use <code>screen</code>, <code>tmux</code> or <code>byobu</code>
(those are linux console/terminal programs) to make your life easier, by keeping
the servers running in background while multiplexing many terminals into a single one.
Instructions on how to use those are not provided here; Please read their manpage
instead or refer to these links:
[https://www.gnu.org/software/screen/manual/ screen] [https://github.com/tmux/tmux/wiki tmux]
[https://www.byobu.org/documentation byobu]
 
Note: MySQL/Mariadb must be running for these commands to work.
 
==== Register an Account via M+ on localserver & GM account ====
 
If everything went well, you should have your servers up and running.
To login on your localhost, you can do it two ways: By our provided tool file,
or by manually adding a new entry to ManaPlus.
 
To use the provided tool file:
 
<code><pre>cd tools/manaplus
./connect_local_server.sh</pre></code>
 
 
To set-up M+ (ManaPlus) to log into localserver:
 
Name: My Dev
 
Address: 127.0.0.1
 
Port: 6901
 
Server type: Evol2
 
Once connected select the register button & sign up for an account
 
Once registered, test by logging into the server.
 
Note: If you have not yet installed client-data you may not see any artwork after logging in.
Note: If you log in to your local server without using the script, Manaplus may use the wrong client-data.
 
Now you should be able to give that account GM by running
<code>make givegm</code>
 
You should now have a working Local server with GM 99 on your account.
 
=== Additional steps you may need for development (Advanced) ===
 
Before next steps you should open terminal in evol-all direcory.
 
You should something like this:
 
<code>$~/evol-all</code>
 
Depend what kind of development you will do, you need run of any additional steps.
 
Also you can mix any of 4.x. steps in one directory.
 
====Server code development ====
 
If you want run or develop server code or server content, you must install "server" task.
 
run command:
 
<code>./init.sh server</code>
 
and you will see repository cloning progress.
 
After will be created new directories: '''server-code''', '''server-data''', '''server-plugin'''
 
==== Run manaplus or client data / art / maps etc. development ====
 
If you want develop client side content, you must install "client" task.
 
run command:
 
<code>./init.sh client</code>
 
and you will see repository cloning progress.
 
After will be created new directories: '''client-data''', '''tools'''
 
If you want latest ManaPlus version, you also should install task "manaplus" and build it. See [[#ManaPlus|1.3.5]].
 
==== Music ====
 
If you want develop music, you must install "music" task.
 
run command:
 
<code>./init.sh music</code>
 
and you will see repository cloning progress.
 
After will be created new directories: '''music'''
 
==== Local server ====
 
run command:
 
<code>./init.sh local</code>
 
and you will see repository cloning progress.
 
After will be created new directories: '''server-local''', '''server-code''', '''server-data''', '''server-plugin'''
 
Probably you also should add task "client" for running ManaPlus. See [[#Run manaplus or client data / art / maps etc. development|1.3.2]].
 
See also [[localserver|local server]]
 
====ManaPlus ====
 
If you want develop ManaPlus or want run ManaPlus from git, you should install "manaplus" task.
 
run command:
 
<code>./init.sh manaplus</code>
 
and you will see repository cloning progress.
 
After will be created new directories: '''manaplus'''
 
====All====
 
If you want all tasks at same time, you must install task "all".
 
run command:
 
<code>./init.sh all</code>
 
and you will see repository cloning progress.
 
After will be created new directories for all possible work.
 
==== Install and configure mariadb server ====
 
For run server you need install and configure mysql or mariadb.
 
[[guidelines/installmariadb|See here]]
 
====Run client and connect to server ====
 
Before running client you must install atleast '''server''' and '''client''' task or '''all''' task.
 
If you want run ManaPlus and connect to local server, run this:
 
<code><pre>cd tools/manaplus
./connect_local_server.sh</pre></code>
 
Client will start and connect to server.
 
==Making Content==
===Making Map Content===
See [[Mapping Tutorial]].
====Adding Map Content to Local Server====
The server have its own version of the collision data of a map.
To update it, on server-data folder, simply run:
 
: <code>make maps</code>
===Adding Art Content===
Use '''<code>dyecmd</code>''' to ensure your artwork is sane (32 bit, PNG, alpha, without any pallete and other data). This command is provided at the ManaPlus package. See the ManaPlus Website for help with XML files. (You may need to look at sitemap)
 
===Adding Music Content===
All music must be OGG Vorbis with 44100 Hz resolution.
 
===ADDING QUESTS & SCRIPTS===
Edit the txt files at "server-data/npc". Use "make maps" to generate the imports, if you added a new NPC.
 
Restart map-server when you change any script, and it'll have effect. You can also use "<code>@serverexit 101</code>" if you're using the test server wrappers; Or use "<code>@reloadnpc</code>" command if you changed a single file. Map-Server will report any errors or warnings on your code. You must not introduce any new error or warning to have your new quest/script accepted.
 
The scripting references can be found at "doc/server/scripts", which should have been included by "init.sh all".
 
===Bit Masking===
Bit Masking helps us reduce our variable count while offering a dynamic method of script writing that is versatile and flexible to both linear and non-linear script writing.
 
[[Dev:Bit_mask_tutorial]]
 
===Generating client updates===
This is NOT necessary if you're using the connect_to_local_server wrapper.
 
* Shutdown all your servers.
 
* First we are going to check the config files to be sure the paths are correct.
cd /evol-all/tools/update/
ls
 
Output:
addmods.sh  adler32.c      commit.txt      createnew.sh  musiccommit.txt  update_music.sh
adler32    commit_old.txt  create_music.sh  files        news.txt        update.sh End of output
 
* The easiest solution is to create a symbolic link to that directory
ln -s <Path to Public Webserver Dir> /evol-all/tools/update/upload/
 
example: ln -s /var/www/updates /evol-all/tools/update/upload/
 
*Save the file and get back in the terminal.
edit /evol-all/server-data/conf/import/login-server.conf
 
Change update_server to reflect the hostname & location of the server updates.
 
TMW's test-server example:
<code>update_server: "http://updates.themanaworld.org/test-updates/"</code>
 
* Now we can make us a update.
cd /evol-all/server-data/
make updates
 
* Each time you want to make new updates
make sure you have a new commit then
cd /evol-all/server-data/
make updates
 
 
== Obsolete (for historical references) ==
===Getting Art Content Into The Game (obsolete)===
* Post [CPT] or [WIP] art on the [http://forums.themanaworld.org/viewforum.php?f=8 graphics development forum].
* Post [CPT] or [WIP] art on the [http://forums.themanaworld.org/viewforum.php?f=8 graphics development forum].
* Work with the Art Director and collaborators on getting the art consistent.
* Work with the Art Director and collaborators on getting the art consistent.
Line 113: Line 323:
::* Art content client data affected: entry added in items.xml, monsters.xml or other identification file.
::* Art content client data affected: entry added in items.xml, monsters.xml or other identification file.


===Handling General Content Team [REQ]s===
====Handling General Content Team [REQ]s (obsolete)====
* General Content Team member adds a [REQ] topic to the graphics development forum.
* General Content Team member adds a [REQ] topic to the graphics development forum.
* General Content Team member adds the [REQ] topic as a hyperlink under "Required Art" on the [[Art in Development]] page.
* General Content Team member adds the [REQ] topic as a hyperlink under "Required Art" on the [[Art in Development]] page.
* Art Director attempts to network with other artists to assign the art to an artist.
* Art Director attempts to network with other artists to assign the art to an artist.


==Bit Masking==
Bit Masking helps us reduce our variable count while offering a dynamic method of script writing that is versatile and flexible to both linear and non-linear script writing.
[http://dl.dropbox.com/u/6520164/tester1.txt Bit Masking Test Script]
=Using the Test Server=
*[[Testing Server]]
=Using The Mana World Workshop=
*[http://locke.suu.edu/~adams/mana/workshop/manual/ The Mana World Workshop Manual]


=Trouble Shooting=
[[Category:Development]]
see [http://wiki.themanaworld.org/index.php/Setting_up_eAthena_on_Linux#Troubleshooting for Linux Troubleshooting]

Latest revision as of 03:56, 27 March 2024

Developing for The Mana World

Note: The links here are for the rEvolt Project.

For TMW Classic, you should replace gitlab.com/evol with git.themanaworld.org/evolved, keeping other parts intact.

Quick Start for local server

Install Dependencies

Depending on what sql and OS your using install Maria or Mysql packages:

* Maria packages: mariadb-server, mariadb-client, mariadbclient-dev, libmariadbclient-dev
* Mysql packages: mysqldb-server, mysqldb-client, mysqlclient-dev, libmysqlclient-dev

All these other dependencies need to be installed regardless of sql used:

* zlib-dev or zlib1g-dev package, libpcre3-dev, gcc, make, automake, autoconf, libtool

Some distributions also require manual installation of ssl and crypto packages:

* libssl-dev, libcrypto++-dev

Note: Some packages may named differently depending on your distro.
You may use apt-cache search libpcre to check all packages which contain 'libpcre' on the name, and hopefully find the right one. It may be named libpcre0v5-dev for example. You may try with libpcre++-dev first.

apt-cache is not available on non-Debian or non-Debian-based distros.
You may want to use apt search <package(s)> instead.

You need gcc <= 9 to compile the servers. gcc-10 and later will raise errors when compiling md5 functions. We advise to install gcc-9 packages and pass it under CC environment variable.

Open command line/terminal

~$

Clone repository

run command:

git clone https://git.themanaworld.org/revolt/all.git

You will see something like this:

$ git clone https://git.themanaworld.org/revolt/all.git
Cloning into 'evol-all'...
remote: Counting objects: 69, done.
remote: Compressing objects: 100% (67/67), done.
remote: Total 69 (delta 36), reused 0 (delta 0)
Receiving objects: 100% (69/69), 7.05 KiB | 0 bytes/s, done.
Resolving deltas: 100% (36/36), done.
Checking connectivity... done.

Note: For TMW Classic, you should run this command instead: git clone https://git.themanaworld.org/evolved/all.git

Everything else should be roughly the same afterward.

Init repos

In evol-all directory run command

./init.sh all

Build and set-up local server

In directory client-data in evol-all run the following command:

ln -s ../music/music music

If you want to be able to listen to background music.


In directory server-data in evol-all run the following command:

make new

On multi-core/thread CPUs try appending -j<number of cores> to delegate a task to every core/thread like make new -j4

(Note: Mariadb/MySQL needs to be up and running for this command to execute correctly, Debian/Ubuntu may automatically do this setup on installation. Read your distro's manuals for SQL setup and configuration.)

Running server

You need to repeat this step every time your computer shuts down, or every time you close your terminal.

First, you should be in server-data folder.

In three separate console windows, run the following commands:

./login-server
./char-server
./map-server

Pay attention to any bug reported in any of them, as they often mean something went wrong in the previous steps.

Keep in mind that if you close the terminal windows, you'll need to run these commands again. You should preferably run them in this order.

Note: You may also use screen, tmux or byobu (those are linux console/terminal programs) to make your life easier, by keeping the servers running in background while multiplexing many terminals into a single one. Instructions on how to use those are not provided here; Please read their manpage instead or refer to these links: screen tmux byobu

Note: MySQL/Mariadb must be running for these commands to work.

Register an Account via M+ on localserver & GM account

If everything went well, you should have your servers up and running. To login on your localhost, you can do it two ways: By our provided tool file, or by manually adding a new entry to ManaPlus.

To use the provided tool file:

cd tools/manaplus
./connect_local_server.sh


To set-up M+ (ManaPlus) to log into localserver:

Name: My Dev

Address: 127.0.0.1

Port: 6901

Server type: Evol2

Once connected select the register button & sign up for an account

Once registered, test by logging into the server.

Note: If you have not yet installed client-data you may not see any artwork after logging in. Note: If you log in to your local server without using the script, Manaplus may use the wrong client-data.

Now you should be able to give that account GM by running make givegm

You should now have a working Local server with GM 99 on your account.

Additional steps you may need for development (Advanced)

Before next steps you should open terminal in evol-all direcory.

You should something like this:

$~/evol-all

Depend what kind of development you will do, you need run of any additional steps.

Also you can mix any of 4.x. steps in one directory.

Server code development

If you want run or develop server code or server content, you must install "server" task.

run command:

./init.sh server

and you will see repository cloning progress.

After will be created new directories: server-code, server-data, server-plugin

Run manaplus or client data / art / maps etc. development

If you want develop client side content, you must install "client" task.

run command:

./init.sh client

and you will see repository cloning progress.

After will be created new directories: client-data, tools

If you want latest ManaPlus version, you also should install task "manaplus" and build it. See 1.3.5.

Music

If you want develop music, you must install "music" task.

run command:

./init.sh music

and you will see repository cloning progress.

After will be created new directories: music

Local server

run command:

./init.sh local

and you will see repository cloning progress.

After will be created new directories: server-local, server-code, server-data, server-plugin

Probably you also should add task "client" for running ManaPlus. See 1.3.2.

See also local server

ManaPlus

If you want develop ManaPlus or want run ManaPlus from git, you should install "manaplus" task.

run command:

./init.sh manaplus

and you will see repository cloning progress.

After will be created new directories: manaplus

All

If you want all tasks at same time, you must install task "all".

run command:

./init.sh all

and you will see repository cloning progress.

After will be created new directories for all possible work.

Install and configure mariadb server

For run server you need install and configure mysql or mariadb.

See here

Run client and connect to server

Before running client you must install atleast server and client task or all task.

If you want run ManaPlus and connect to local server, run this:

cd tools/manaplus
./connect_local_server.sh

Client will start and connect to server.

Making Content

Making Map Content

See Mapping Tutorial.

Adding Map Content to Local Server

The server have its own version of the collision data of a map. To update it, on server-data folder, simply run:

make maps

Adding Art Content

Use dyecmd to ensure your artwork is sane (32 bit, PNG, alpha, without any pallete and other data). This command is provided at the ManaPlus package. See the ManaPlus Website for help with XML files. (You may need to look at sitemap)

Adding Music Content

All music must be OGG Vorbis with 44100 Hz resolution.

ADDING QUESTS & SCRIPTS

Edit the txt files at "server-data/npc". Use "make maps" to generate the imports, if you added a new NPC.

Restart map-server when you change any script, and it'll have effect. You can also use "@serverexit 101" if you're using the test server wrappers; Or use "@reloadnpc" command if you changed a single file. Map-Server will report any errors or warnings on your code. You must not introduce any new error or warning to have your new quest/script accepted.

The scripting references can be found at "doc/server/scripts", which should have been included by "init.sh all".

Bit Masking

Bit Masking helps us reduce our variable count while offering a dynamic method of script writing that is versatile and flexible to both linear and non-linear script writing.

Dev:Bit_mask_tutorial

Generating client updates

This is NOT necessary if you're using the connect_to_local_server wrapper.

  • Shutdown all your servers.
  • First we are going to check the config files to be sure the paths are correct.
cd /evol-all/tools/update/
ls
Output:

addmods.sh adler32.c commit.txt createnew.sh musiccommit.txt update_music.sh adler32 commit_old.txt create_music.sh files news.txt update.sh End of output

  • The easiest solution is to create a symbolic link to that directory

ln -s <Path to Public Webserver Dir> /evol-all/tools/update/upload/

example: ln -s /var/www/updates /evol-all/tools/update/upload/

  • Save the file and get back in the terminal.

edit /evol-all/server-data/conf/import/login-server.conf

Change update_server to reflect the hostname & location of the server updates.

TMW's test-server example: update_server: "http://updates.themanaworld.org/test-updates/"

  • Now we can make us a update.
cd /evol-all/server-data/
make updates
  • Each time you want to make new updates
make sure you have a new commit then
cd /evol-all/server-data/
make updates


Obsolete (for historical references)

Getting Art Content Into The Game (obsolete)

  • Post [CPT] or [WIP] art on the graphics development forum.
  • Work with the Art Director and collaborators on getting the art consistent.
  • When finished, edit topic as [FND].
  • Art Director approves [FND] art and marks it [RC]. Should the art need animation or special effect, the Art Director requests xml for it. If not, the Art Director commits it and marks as [GIT].
  • XML Team reviews or drafts xml related to the art, then commits the .png and .xml. Topic can be marked as [GIT].
  • Art content affected: client-data/graphics/sprites then related directory for .png and .xml animation addition.
  • Project developer requests database entry reservations from a General Content Team developer.
  • Map entry server data affected: resnametable.txt and .wlk files in the data directory, map content files in the npc directory.
  • Map entry client data affected: .tmx files in the maps directory.
  • Map entry requires the running of the .tmx converter (aka java converter) before commits can be made.
  • Art content server data affected: entry made in the item_db.txt, mob_db.txt or mob_skills_db.txt in the db directory.
  • Art content client data affected: entry added in items.xml, monsters.xml or other identification file.

Handling General Content Team [REQ]s (obsolete)

  • General Content Team member adds a [REQ] topic to the graphics development forum.
  • General Content Team member adds the [REQ] topic as a hyperlink under "Required Art" on the Art in Development page.
  • Art Director attempts to network with other artists to assign the art to an artist.