Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4989 in Layer 7

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0306. Min Act: -0.0000

Data Index: 382049 (The Pile)

Max Activating Token Index: 974

Text #1

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0334. Min Act: -0.0000

Data Index: 1266060 (The Pile)

Max Activating Token Index: 944

Text #2

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0308. Min Act: -0.0000

Data Index: 539220 (The Pile)

Max Activating Token Index: 943

Text #3

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0330. Min Act: -0.0000

Data Index: 1738927 (The Pile)

Max Activating Token Index: 998

Text #4

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0298. Min Act: -0.0000

Data Index: 1802297 (The Pile)

Max Activating Token Index: 633

Text #5

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0248. Min Act: -0.0000

Data Index: 1021203 (The Pile)

Max Activating Token Index: 972

Text #6

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0298. Min Act: -0.0000

Data Index: 838000 (The Pile)

Max Activating Token Index: 1016

Text #7

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0315. Min Act: -0.0000

Data Index: 804542 (The Pile)

Max Activating Token Index: 1015

Text #8

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0280. Min Act: -0.0000

Data Index: 1947727 (The Pile)

Max Activating Token Index: 1007

Text #9

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0293. Min Act: -0.0000

Data Index: 1082132 (The Pile)

Max Activating Token Index: 639

Text #10

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0225. Min Act: -0.0000

Data Index: 254696 (The Pile)

Max Activating Token Index: 940

Text #11

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0322. Min Act: -0.0000

Data Index: 1637154 (The Pile)

Max Activating Token Index: 342

Text #12

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0225. Min Act: -0.0000

Data Index: 939641 (The Pile)

Max Activating Token Index: 1013

Text #13

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0256. Min Act: -0.0000

Data Index: 93508 (The Pile)

Max Activating Token Index: 965

Text #14

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0210. Min Act: -0.0000

Data Index: 726915 (The Pile)

Max Activating Token Index: 978

Text #15

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0226. Min Act: -0.0000

Data Index: 1319535 (The Pile)

Max Activating Token Index: 968

Text #16

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0236. Min Act: -0.0000

Data Index: 1979401 (The Pile)

Max Activating Token Index: 457

Text #17

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0214. Min Act: -0.0000

Data Index: 204721 (The Pile)

Max Activating Token Index: 1003

Text #18

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0212. Min Act: -0.0000

Data Index: 910460 (The Pile)

Max Activating Token Index: 320

Text #19

Max Range: 0.0334. Min Range: -0.0334

Max Act: 0.0194. Min Act: -0.0000

Data Index: 747686 (The Pile)

Max Activating Token Index: 926