Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4652 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.2403. Min Act: -0.0000

Data Index: 167244 (The Pile)

Max Activating Token Index: 229

Text #1

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.1080. Min Act: -0.0000

Data Index: 896213 (The Pile)

Max Activating Token Index: 812

Text #2

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.1043. Min Act: -0.0000

Data Index: 167319 (The Pile)

Max Activating Token Index: 439

Text #3

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.1026. Min Act: -0.0000

Data Index: 1734701 (The Pile)

Max Activating Token Index: 404

Text #4

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0930. Min Act: -0.0000

Data Index: 1840752 (The Pile)

Max Activating Token Index: 769

Text #5

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0970. Min Act: -0.0000

Data Index: 607410 (The Pile)

Max Activating Token Index: 809

Text #6

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0956. Min Act: -0.0000

Data Index: 269753 (The Pile)

Max Activating Token Index: 125

Text #7

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.1099. Min Act: -0.0000

Data Index: 1252245 (The Pile)

Max Activating Token Index: 225

Text #8

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0844. Min Act: -0.0000

Data Index: 345161 (The Pile)

Max Activating Token Index: 230

Text #9

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0865. Min Act: -0.0000

Data Index: 423454 (The Pile)

Max Activating Token Index: 700

Text #10

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0863. Min Act: -0.0000

Data Index: 1170010 (The Pile)

Max Activating Token Index: 136

Text #11

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0761. Min Act: -0.0000

Data Index: 1730685 (The Pile)

Max Activating Token Index: 614

Text #12

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0777. Min Act: -0.0000

Data Index: 1757225 (The Pile)

Max Activating Token Index: 804

Text #13

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0836. Min Act: -0.0000

Data Index: 778416 (The Pile)

Max Activating Token Index: 721

Text #14

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0904. Min Act: -0.0000

Data Index: 674900 (The Pile)

Max Activating Token Index: 872

Text #15

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0785. Min Act: -0.0000

Data Index: 760889 (The Pile)

Max Activating Token Index: 535

Text #16

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0800. Min Act: -0.0000

Data Index: 382854 (The Pile)

Max Activating Token Index: 714

Text #17

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0803. Min Act: -0.0000

Data Index: 18212 (The Pile)

Max Activating Token Index: 500

Text #18

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0799. Min Act: -0.0000

Data Index: 590920 (The Pile)

Max Activating Token Index: 1011

Text #19

Max Range: 0.2403. Min Range: -0.2403

Max Act: 0.0811. Min Act: -0.0000

Data Index: 1362382 (The Pile)

Max Activating Token Index: 407