Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 197 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.2791. Min Act: -0.0000

Data Index: 860234 (The Pile)

Max Activating Token Index: 964

Text #1

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.1239. Min Act: -0.0000

Data Index: 1429777 (The Pile)

Max Activating Token Index: 948

Text #2

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.1034. Min Act: -0.0000

Data Index: 1819094 (The Pile)

Max Activating Token Index: 598

Text #3

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.1159. Min Act: -0.0000

Data Index: 1935172 (The Pile)

Max Activating Token Index: 200

Text #4

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.0542. Min Act: -0.0000

Data Index: 1480775 (The Pile)

Max Activating Token Index: 590

Text #5

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.9209. Min Act: -0.0000

Data Index: 1670885 (The Pile)

Max Activating Token Index: 632

Text #6

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.9443. Min Act: -0.0000

Data Index: 1435376 (The Pile)

Max Activating Token Index: 875

Text #7

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.0382. Min Act: -0.0000

Data Index: 327093 (The Pile)

Max Activating Token Index: 790

Text #8

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.9510. Min Act: -0.0000

Data Index: 1215576 (The Pile)

Max Activating Token Index: 753

Text #9

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8640. Min Act: -0.0000

Data Index: 441918 (The Pile)

Max Activating Token Index: 502

Text #10

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.9818. Min Act: -0.0000

Data Index: 1247020 (The Pile)

Max Activating Token Index: 894

Text #11

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.9130. Min Act: -0.0000

Data Index: 1259672 (The Pile)

Max Activating Token Index: 238

Text #12

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.0945. Min Act: -0.0000

Data Index: 46665 (The Pile)

Max Activating Token Index: 240

Text #13

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8412. Min Act: -0.0000

Data Index: 607261 (The Pile)

Max Activating Token Index: 833

Text #14

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8549. Min Act: -0.0000

Data Index: 1349126 (The Pile)

Max Activating Token Index: 128

Text #15

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8800. Min Act: -0.0000

Data Index: 88191 (The Pile)

Max Activating Token Index: 57

Text #16

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8034. Min Act: -0.0000

Data Index: 950946 (The Pile)

Max Activating Token Index: 698

Text #17

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8286. Min Act: -0.0000

Data Index: 641997 (The Pile)

Max Activating Token Index: 372

Text #18

Max Range: 1.2791. Min Range: -1.2791

Max Act: 1.0470. Min Act: -0.0000

Data Index: 92088 (The Pile)

Max Activating Token Index: 887

Text #19

Max Range: 1.2791. Min Range: -1.2791

Max Act: 0.8162. Min Act: -0.0000

Data Index: 1799214 (The Pile)

Max Activating Token Index: 566