Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2450 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.4126. Min Act: -0.0000

Data Index: 528502 (The Pile)

Max Activating Token Index: 75

Text #1

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.3600. Min Act: -0.0000

Data Index: 210499 (The Pile)

Max Activating Token Index: 703

Text #2

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.3067. Min Act: -0.0000

Data Index: 1431408 (The Pile)

Max Activating Token Index: 547

Text #3

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2941. Min Act: -0.0000

Data Index: 1594959 (The Pile)

Max Activating Token Index: 988

Text #4

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2736. Min Act: -0.0000

Data Index: 12051 (The Pile)

Max Activating Token Index: 696

Text #5

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2978. Min Act: -0.0000

Data Index: 1207820 (The Pile)

Max Activating Token Index: 774

Text #6

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2927. Min Act: -0.0000

Data Index: 1134525 (The Pile)

Max Activating Token Index: 327

Text #7

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.3409. Min Act: -0.0000

Data Index: 1253398 (The Pile)

Max Activating Token Index: 616

Text #8

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2854. Min Act: -0.0000

Data Index: 828858 (The Pile)

Max Activating Token Index: 62

Text #9

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2691. Min Act: -0.0000

Data Index: 1802980 (The Pile)

Max Activating Token Index: 624

Text #10

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2663. Min Act: -0.0000

Data Index: 1195927 (The Pile)

Max Activating Token Index: 137

Text #11

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.3428. Min Act: -0.0000

Data Index: 1744463 (The Pile)

Max Activating Token Index: 725

Text #12

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2397. Min Act: -0.0000

Data Index: 873498 (The Pile)

Max Activating Token Index: 620

Text #13

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2528. Min Act: -0.0000

Data Index: 1732445 (The Pile)

Max Activating Token Index: 340

Text #14

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2425. Min Act: -0.0000

Data Index: 1383973 (The Pile)

Max Activating Token Index: 398

Text #15

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2548. Min Act: -0.0000

Data Index: 1269740 (The Pile)

Max Activating Token Index: 502

Text #16

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2225. Min Act: -0.0000

Data Index: 1006484 (The Pile)

Max Activating Token Index: 575

Text #17

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2160. Min Act: -0.0000

Data Index: 1102304 (The Pile)

Max Activating Token Index: 655

Text #18

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2201. Min Act: -0.0000

Data Index: 1711741 (The Pile)

Max Activating Token Index: 200

Text #19

Max Range: 0.4126. Min Range: -0.4126

Max Act: 0.2252. Min Act: -0.0000

Data Index: 1325305 (The Pile)

Max Activating Token Index: 608