Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 2873 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.2887. Min Act: -0.0000

Data Index: 1540957 (The Pile)

Max Activating Token Index: 150

Text #1

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.2014. Min Act: -0.0000

Data Index: 1902590 (The Pile)

Max Activating Token Index: 909

Text #2

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.3268. Min Act: -0.0000

Data Index: 8520 (The Pile)

Max Activating Token Index: 323

Text #3

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.3734. Min Act: -0.0000

Data Index: 1409385 (The Pile)

Max Activating Token Index: 1009

Text #4

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.3336. Min Act: -0.0000

Data Index: 1191935 (The Pile)

Max Activating Token Index: 240

Text #5

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.8851. Min Act: -0.0000

Data Index: 101519 (The Pile)

Max Activating Token Index: 918

Text #6

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.0824. Min Act: -0.0000

Data Index: 1618575 (The Pile)

Max Activating Token Index: 685

Text #7

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.9289. Min Act: -0.0000

Data Index: 12914 (The Pile)

Max Activating Token Index: 627

Text #8

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.1142. Min Act: -0.0000

Data Index: 1408841 (The Pile)

Max Activating Token Index: 933

Text #9

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.0556. Min Act: -0.0000

Data Index: 1671200 (The Pile)

Max Activating Token Index: 932

Text #10

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.2848. Min Act: -0.0000

Data Index: 332911 (The Pile)

Max Activating Token Index: 625

Text #11

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.1781. Min Act: -0.0000

Data Index: 964021 (The Pile)

Max Activating Token Index: 727

Text #12

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.0767. Min Act: -0.0000

Data Index: 1181977 (The Pile)

Max Activating Token Index: 339

Text #13

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.0654. Min Act: -0.0000

Data Index: 1494371 (The Pile)

Max Activating Token Index: 103

Text #14

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.9735. Min Act: -0.0000

Data Index: 149883 (The Pile)

Max Activating Token Index: 262

Text #15

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.9705. Min Act: -0.0000

Data Index: 1601897 (The Pile)

Max Activating Token Index: 540

Text #16

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.0727. Min Act: -0.0000

Data Index: 403959 (The Pile)

Max Activating Token Index: 555

Text #17

Max Range: 1.3734. Min Range: -1.3734

Max Act: 1.2164. Min Act: -0.0000

Data Index: 223032 (The Pile)

Max Activating Token Index: 727

Text #18

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.9050. Min Act: -0.0000

Data Index: 1193702 (The Pile)

Max Activating Token Index: 931

Text #19

Max Range: 1.3734. Min Range: -1.3734

Max Act: 0.9086. Min Act: -0.0000

Data Index: 1876804 (The Pile)

Max Activating Token Index: 767