Model: SoLU Model: 10 Layers, 5120 Neurons per Layer

Dataset: The Pile

Neuron 4740 in Layer 8

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-10l-pile')

Text #0

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3457. Min Act: -0.0000

Data Index: 1646221 (The Pile)

Max Activating Token Index: 198

Text #1

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3363. Min Act: -0.0000

Data Index: 239394 (The Pile)

Max Activating Token Index: 270

Text #2

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3204. Min Act: -0.0000

Data Index: 75351 (The Pile)

Max Activating Token Index: 799

Text #3

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2988. Min Act: -0.0000

Data Index: 8068 (The Pile)

Max Activating Token Index: 201

Text #4

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3244. Min Act: -0.0000

Data Index: 8733 (The Pile)

Max Activating Token Index: 365

Text #5

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3213. Min Act: -0.0000

Data Index: 1601333 (The Pile)

Max Activating Token Index: 512

Text #6

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2996. Min Act: -0.0000

Data Index: 1962998 (The Pile)

Max Activating Token Index: 551

Text #7

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.3373. Min Act: -0.0000

Data Index: 1041548 (The Pile)

Max Activating Token Index: 185

Text #8

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2784. Min Act: -0.0000

Data Index: 1076885 (The Pile)

Max Activating Token Index: 330

Text #9

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2759. Min Act: -0.0000

Data Index: 1264597 (The Pile)

Max Activating Token Index: 752

Text #10

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2994. Min Act: -0.0000

Data Index: 566130 (The Pile)

Max Activating Token Index: 606

Text #11

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2892. Min Act: -0.0000

Data Index: 579633 (The Pile)

Max Activating Token Index: 53

Text #12

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2794. Min Act: -0.0000

Data Index: 443377 (The Pile)

Max Activating Token Index: 656

Text #13

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2845. Min Act: -0.0000

Data Index: 853252 (The Pile)

Max Activating Token Index: 930

Text #14

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2957. Min Act: -0.0000

Data Index: 1739964 (The Pile)

Max Activating Token Index: 673

Text #15

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2953. Min Act: -0.0000

Data Index: 990036 (The Pile)

Max Activating Token Index: 417

Text #16

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2975. Min Act: -0.0000

Data Index: 1772475 (The Pile)

Max Activating Token Index: 932

Text #17

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2797. Min Act: -0.0000

Data Index: 379967 (The Pile)

Max Activating Token Index: 392

Text #18

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2834. Min Act: -0.0000

Data Index: 1922910 (The Pile)

Max Activating Token Index: 772

Text #19

Max Range: 0.3457. Min Range: -0.3457

Max Act: 0.2643. Min Act: -0.0000

Data Index: 1550472 (The Pile)

Max Activating Token Index: 167